Improving Sensory Experience in Virtual Reality

November 27, 2018

To date, much of the innovation we’ve seen with virtual reality (VR) has focused on optics and the ability to create believable landscapes for users to interact with. But researchers worldwide aren’t stopping there; strides are being made to create more believable experiences for touch, sound and smell as well.

At Ecole Polytechnique Fédérale de Lausanne in Switzerland, researchers are focused on the importance of passive haptic feedback in making simulations feel believable. At the most basic level, this centers on the experience of touching one’s own body while controlling an avatar: “When carefully treated, the body may act as a free source of high fidelity haptic feedback; if neglected however, it will cause inconsistencies that will lessen the impact of any full-body tracked application.”

Making the experience of self-contact consistent has an outsize effect on making the virtual feel real, since people are more sensitive to inconsistencies than they are to precise movements, the researchers say. As motion capture devices become better and cheaper, this should be increasingly achievable.

Part of being truly immersed in a simulation is having it sound realistic. At Chung-Ang University in South Korea, work is being done to build wireless speakers that can detect their position in a speaker setup and adjust their output accordingly. Using a speaker, a Raspberry Pi and a beacon, the author was able to get great accuracy with the speakers 3-4 meters apart. While surround-sound systems obviously exist already, this would allow for simple setup anywhere, eliminate wires and make the experience safer and more convincing for the user.

Lastly, in what’s perhaps the most radical approach, researchers in Malaysia are working on digitizing smell so that it can be replicated anywhere. Moving away from cartridges that contain chemicals, this approach uses electric current delivered directly to the nasal cells, which causes the recipient to perceive odors that aren’t there. While this was an uncomfortable and early-stage proof of concept, strides in this area would be exciting for VR and many different experiences online.

Sight remains of particular importance, so it’s still worth mentioning here. Researchers from Deakin University in Australia looking at the effects of virtual environments on our mental and cognitive states wrote a paper assessing visual fatigue from VR, which they say is more mental than muscular, since the brain gets overloaded with conflicting signals. While they faced a number of challenges in this pursuit, the work stands to have important implications for improving headset and display design.

To hear more on the latest breakthroughs in the world of VR, watch the special video series on our VR/MR page.