How does virtual reality work?

Imagine if you could go to the cinema and truly immerse yourself in the movie, rather than just have a 3D Iron Man flying out the screen towards you? How about going on holiday without leaving your armchair, playing dangerous sports with no risk of injury or taking your videogames to an entirely new level? This has been the promise of virtual reality (VR) since its inception back in the Fifties – and it’s starting to be realised with new domestic technology that is within reach of everyone.

Virtual reality isn’t like 3D which merely gives the appearance of depth on a flat screen; it actually places you within a 360-degree environment that your senses tell you is real.
Remember, what your eyes see or your fingers touch is just electrical impulses interpreted by your brain. VR essentially works by tricking your senses into believing that they are experiencing a real environment when, in fact, it is completely computer generated.

Other than entertainment, virtual reality has many other applications. The US Army has adopted what it calls its Close Combat Tactical Trainer (CCTT), which uses VR to train soldiers by creating fighting avatars. The system is rather like a cross between the videogame Call Of Duty and Star Trek’s holodeck. Plus, VR is also used by NASA and by medical professionals for advanced surgical training.

Army virtual reality
Soldiers can use virtual reality headsets for training

For VR to work convincingly, you have to feel that you are completely immersed in an environment, but also have the ability to interact with it. It’s the interaction – known as telepresence – that sets VR apart from other virtual world systems and 3D cinema. Also, for VR environments to be perceived as real, feedback has to be present. Think about how you interact with the real world. You can touch and pick up objects, feel their texture and influence things when you open a door or pick up your remote. This is often referred to as force feedback. If you’re a gamer, vibrating controllers and body armour that enables you to feel a bullet hit are good examples of force feedback that are collectively called haptic systems. These are essential components of building a realistic VR environment that will fool your brain.

VR uses a number of technologies together to deliver a convincing world to the explorer. The head mounted display (HMD) is the most important, and uses a technique called stereoscopy that feeds slightly different images to each eye. The Oculus Rift uses a single LCD screen with a colour depth of 24 bits per pixel and includes a 1,000-Hertz adjacent reality tracker that reduces latency to improve the overall quality of the images as the HMD moves. Lenses in front of each eye give the appearance of depth and mean the images totally encompass the wearer’s field of vision.

Oculus Rift teardown
Oculus Rift teardown

Tracking software built into the HMD means that as the user moves their head, the virtual world moves with them. A built-in three-axis gyroscope, magnetometers (which measure the strength of the Earth’s magnetic field) and accelerometers (to measure how fast the HMD is moving in space) all allow accurate head tracking and therefore the perception that the environment is real.

These three technologies change the images fed to the HMD as the wearer moves around. This sense that the scene they see moves as they do is how VR tricks your brain into thinking you are in a real place. Movement is also a key component of convincing VR. The Omni from Virtuix is the perfect companion for a VR HMD and controller. The treadmill works with the user wearing bespoke pinned shoes. When the shoes make contact with the grooved, low- friction surface of the Omni, the plates within the treadmill move to mimic the walking on a flat surface. Users can even run or jump with their relative position fed back to the computer generating the images shown on the HMD. As users walk unsupported – just as you would in the real world – the illusion of movement through the VR space is assured.

Virtuix Omni
The Virtuix Omni virtual reality platform

Of course, a computer-generated environment is just that, so how does a VR system ensure you suspend your disbelief and react to the VR world as if it were real? A convincing VR environment must have graphics of a high enough resolution with images fed to the HMD at around 30 frames per second for it to be believable. A precise combination of texture, shading and lighting effects are all needed to generate a lifelike world. Also, sound should be directional and immerse the user in order to make the audio experience equally as convincing. With hi-res images, haptic systems and surround sound, stand on the edge of a virtual cliff and you will feel your heart rate rise – not recommended for anyone with vertigo!

Tricking the brain

Virtual reality gaming
Virtual reality is set to be big for gaming

VR works as the brain uses systems including proprioception (the sense of where limbs are in space) and how the eyes orientate to the scene when the head moves. Also, place cells in the hippocampus have been shown to be the centre for self-location, which is used by the brain when assessing where the body is in space.

Ultimately, the brain has to believe the concept of presence, which is based on the brain’s past experience of what it feels like to walk down a street, for instance, and how this compares to the virtual street portrayed in the VR environment. The brain compares these past experiences with the CG environment and decides how real it actually is. Also, when virtual limbs are created in the virtual world, the brain is surprisingly willing to merge the fake with the real. Indeed, in one test VR users pulled their arms away from a virtual fire.

Virtual nausea

Virtual reality headset
VR systems can cause motion sickness for some people

Latency is the single biggest barrier to building realistic virtual environments. If the images seen through the HMD are not redrawn (ie rendered) quickly enough, the illusion of reality is shattered. Oculus Rift gets around this issue with what the developers call ‘predictive tracking’.

This technology guesses where the user might look next and gets a head-start on rendering the environment. Latency must be minimised at below 50 milliseconds if the brain isn’t to detect any lag in the images. Early VR systems caused motion sickness in some users. This can’t be completely cured in some people, but VR systems today have hardware and software that work together to minimise large changes in focus depth or vergence (how eyes focus on an object), which can cause nausea.

Discover more amazing technology in the latest issue of How It Works magazine. It’s available from all good retailers, or you can order it online from the ImagineShop. If you have a tablet or smartphone, you can also download the digital version onto your iOS or Android device. To make sure you never miss an issue of How It Works magazine, make sure you subscribe today!

Plus take a look at:

US Army virtual reality tech

Why do 3D movies look washed out?

3D camera history