Few people in the world know more about virtual reality than Michael Abrash.
A celebrated game programmer and technical writer who’s previously worked at Microsoft and Valve, Abrash is now chief scientist at Facebook-owned Oculus, which is finally getting to release the first consumer version of its Oculus Rift virtual reality headset.
On Thursday, at Oculus’ annual developer conference Oculus Connect, Abrash took the stage to close out the presentation by offering a few anecdotes and a ton of information about the current state of virtual reality, and where it needs to go in order to be truly great.
“Imagine a world that feels just as real as this one,” Abrash said. “Imagine being able to believe someone’s virtual avatar is really them.”
Getting to the next level of virtual reality, Abrash said, will require coordinated advances in several different technologies. Specifically, Abrash believes the future of virtual reality will be built on three pillars: driving the human perceptual system, sensing and reconstructing reality, and interaction.
To explain why the human perceptual system is so important, Abrash showed a picture of a car sitting under a roof in front of a mirror. But for some reason, the roof looked different in the mirror.
When the roof was removed and examined, the audience could see its strange shape, which explained why the roof could look so different as it was being rotated.
“The real world is what we see it is,” Abrash said. “The better we can drive the perceptual system, the better the experience.”
Abrash broke down all the human senses to explain how Oculus is identifying the obstacles in preventing virtual reality from feeling real. But of all the senses, the only two we can drive right now are vision and hearing. Taste, smell, touch and the vestibular — our inner ears, our built-in gyroscopes and accelerometers that sense changes in direction and orientation — are still problems that lack any traction.
The biggest problem with trying to replicate senses to manipulate human perception is that improving one aspect can detract from another. Abrash used the example of ergonomics: achieving a wide field of view with dynamic range interferes with the goal of creating a small, lightweight headset that can be worn all day.
But senses are just one problem Oculus needs to tackle: The company is also researching perceptual psychology and neuroscience to devise creative solutions to the hardware and software limitations.
The second pillar Oculus is attempting to tackle is the need to sense and reconstruct the real world in virtual reality. Right now, Oculus can track the headset and controllers, but everyone’s goal in VR is to see their hands and their whole bodies. People want to see other people in VR and feel that they’re real, not just virtual avatars. The huge challenge is to track faces, eyes, hands, and hair, and reconstruct it all in real time. Of course, there are a ton of different limitations, from the sensors to the fact you’re trying to track millions of moving objects simultaneously.
“Making it work will require looking at the whole reconstruction stack from the ground up, from the hardware to the software,” Abrash said.
Once Oculus has the ability to drive the senses to produce experiences that feel real — and pull other people into those experiences — the third and final pillar require interacting with that virtual world. Oculus Touch, the half-moon shaped sensors you hold while wearing the Rift headset, is a big step in that direction. But Abrash says he wants hands to be the dexterous virtual manipulators they are in the real world.
That means probably not using a solution like this:
Using your hands in virtual reality is incredibly hard: Abrash says “there’s no hope of producing the haptics of the real world because there’s no feasible way to reproduce real world kinematics.” In other words, when you put your hand on a virtual table, there’s no technology that can stop your hand from going through it. A new kind of haptic technology based on touch needs to be developed so the hands can sense and respond to virtual interactions.
Still, despite the monumental challenges presented by these three pillars needed to make VR feel real, researching and exploring this space will ultimately be a team effort. It will require interdisciplinary research in optics, sensors, engineering, programming and much more.
“There’s the popular myth of the brilliant flash of genius that solves a problem effortlessly,” Abrash said. “But in reality, it’s teamwork and trying new things.”
Business Insider Emails & Alerts
Site highlights each day to your inbox.