In the ever-evolving landscape of virtual reality (VR) technology, a number of key hurdles remain. But a team of computer scientists have tackled one of the major challenges in VR that will greatly improve user experience — enabling an immersive virtual experience while being physically limited to one’s actual, real-world space.

---

A user wears the researchers’ experimental setup — a Vive HMD augmented with SMI gaze tracking. Superimposed are the top view of the recorded movements of the physical path in a 3.5 m × 3.5 m real room and the virtual path in a much larger 6.4 m × 6.4 m synthetic space. The team demonstrates that saccades can significantly increase the rotation gains during redirection without introducing visual distortions or simulator sickness. Their new method can be applied to large, open virtual spaces and small physical environments for room-scale VR.

 

In the ever-evolving landscape of virtual reality (VR) technology, a number of key hurdles remain. But a team of computer scientists have tackled one of the major challenges in VR that will greatly improve user experience — enabling an immersive virtual experience while being physically limited to one’s actual, real-world space. The research team will present their work at SIGGRAPH 2018.

Computer scientists from Stony Brook University, NVIDIA and Adobe have collaborated on a computational framework that gives VR users the perception of infinite walking in the virtual world — while limited to a small physical space. The framework also enables this free-walking experience for users without causing dizziness, shakiness, or discomfort typically tied to physical movement in VR. And, users avoid bumping into objects in the physical space while in the VR world.

To do this, the researchers focused on manipulating a user’s walking direction by working with a basic natural phenomenon of the human eye, called saccade. Saccades are quick eye movements that occur when we look at a different point in our field of vision, like when scanning a room or viewing a painting. Saccades occur without our control and generally several times per second. During that time, our brains largely ignore visual input in a phenomenon known as “saccadic suppression” — leaving us completely oblivious to our temporary blindness, and the motion that our eyes performed.

“In VR, we can display vast universes; however, the physical spaces in our homes and offices are much smaller,” says lead author of the work, Qi Sun, a PhD student at Stony Brook University and former research intern at Adobe Research and NVIDIA. “It’s the nature of the human eye to scan a scene by moving rapidly between points of fixation. We realized that if we rotate the virtual camera just slightly during saccades, we can redirect a user’s walking direction to simulate a larger walking space.”

Using a head- and eye-tracking VR headset, the researchers’ new method detects saccadic suppression and redirects users during the resulting temporary blindness. When more redirection is required, researchers attempt to encourage saccades using a tailored version of subtle gaze direction — a method that can dynamically encourage saccades by creating points of contrast in our visual periphery.

The team who authored the research, titled “Towards Virtual Reality Infinite Walking: Dynamic Saccade Redirection,” will present their work at SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia. The annual conference and exhibition showcases the world’s leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

To date, existing methods addressing infinite walking in VR have limited redirection capabilities or cause undesirable scene distortions; they have also been unable to avoid obstacles in the physical world, like desks and chairs. The team’s new method dynamically redirects the user away from these objects. The method runs fast, so it is able to avoid moving objects as well, such as other people in the same room.

The researchers ran user studies and simulations to validate their new computational system, including having participants perform game-like search and retrieval tasks. Overall, virtual camera rotation was unnoticeable to users during episodes of saccadic suppression; they could not tell that they were being automatically redirected via camera manipulation. Additionally, in testing the team’s method for dynamic path planning in real-time, users were able to walk without running into walls and furniture, or moving objects like fellow VR users.

“Currently in VR, it is still difficult to deliver a completely natural walking experience to VR users,” says Sun. “That is the primary motivation behind our work — to eliminate this constraint and enable fully immersive experiences in large virtual worlds.”

Though mostly applicable to VR gaming, the new system could potentially be applied to other industries, including architectural design, education, and film production.

An Israeli grandmother glimpses herself as a Palestinian teen. A star athlete experiences what life would be like in a wheelchair.

These are not plots of dystopian movies. They are experiences that take place in virtual reality, which technologists believe will be the next major platform for everything from gaming to social interaction and perhaps even global diplomacy.

Marketers predict VR headsets will soon top wish lists for kids and young adults from the Silicon Valley to Hong Kong.

The computer-generated images beamed to devices strapped around a person’s head allow users to experience “presence”—the sense that they’re entering video games or movies, climbing a treacherous Vietnamese mountain or scuba diving at a coral reef.

Potential benefits include hands-on teaching with a classroom of far-flung students, or holding a business meeting whose global participants sense they’re rubbing elbows.

The upcoming rollout of the Oculus Rift—a $599 headset offering studio-quality VR to the general public—is expected to jump-start industry sales.

Sony meanwhile announced at this week’s Game Developers Conference in San Francisco it would launch its PlayStation VR headgear priced at $399 in October. Many others have VR equipment hitting the market.

Along with its cousin, augmented reality, VR is forecast as a huge market that could push aside smart phones and computer tablets.

From a California garage

VR has been a dream of futurists and tech geeks for decades. But until recently, devices were relegated to research labs because of their exorbitant cost, clunky construction and quality issues that included motion sickness.

At Stanford University’s Virtual Human Interaction Lab, experiments were done until 2014 with a $40,000 device that gave users neck aches; now the lab uses a lightweight Rift at a fraction of the cost.

“I believe in virtual reality and I believed it could be amazing, but that was not a view that was shared by everyone,” Rift inventor Palmer Luckey said.

At the San Francisco conference, users pivoted to shoot would-be attackers and flinched at imaginary flying objects.

“Vision is really important. You rely on it for a majority of your senses,” said Jason Rubin, who as head of worldwide studios oversees content development for Oculus.

“So if we can take over your eyes, we can get control of your belief system.”

Oculus, bought by Facebook in 2014 for $2 billion, is competing with companies such as Google, Samsung and Sony in creating VR devices, with analysts expecting sales of 12 million headsets by the end of this year.

‘Bigger, more disruptive’

But Tim Merel, founder of technology advisory firm Digi-Capital, says VR will be eclipsed by augmented reality, or AR, within a few years.

VR is fully immersive, meaning a user can’t walk down a street wearing a headset. AR is partly immersive: a person can do everyday tasks while augmenting them with virtual images, using holograms (such as flying dinosaurs) superimposed on the user’s field of vision.

While Merel thinks VR will cannibalize video games and become a $30 billion market by 2020, he sees AR as taking over the smartphone and tablet market and accounting for $90 billion in annual sales in the same period.

“Our broad view is that AR will be bigger, more disruptive and faster in terms of its effects than mobile was compared to the original Internet,” Merel said.

While most VR content now focuses on gaming, it has the potential to impact everything from architecture to military training to travel.

Developers envision its use in dealing with phobias and addiction, or in helping youngsters combat bullying. The United Nations is using a VR film to give people a sense of living in a Syrian refugee camp. The New York Times and others are using VR films for immersive news reports.

Jeremy Bailenson, a Stanford professor of communication who founded the lab, said school children might use VR for empathy training. But he acknowledged limits.

“Could this work in the Mideast conflict? I don’t know,” he said. “It’s not a magic bullet or anything.”

There also are potential risks, such as overuse or people discovering they’re more comfortable in a virtual world.

“When porn feels like sex, how does that affect reproduction rates?” Bailenson asked.

The Rift, about the size of a brick but considerably lighter, will be shipped March 28 to customers who pre-ordered it. Oculus is not yet saying when the device will be available in stores. Many users will need a new computer to run the Rift, potentially tripling the $599 price.

Luckey, who attended the developers conference in a Hawaiian shirt, shorts and flip-flops, acknowledged the Rift is still too expensive and limited in its capabilities, but that with improvements “it is going to go well beyond being a toy.”

“I think it’s going to be the next smartphone and the last smartphone. Once you perfect virtual reality, there’s no reason to create anything else,” he said. “I see people continuously moving between the real world and the virtual world.”