I'd say it's primarily caused by a disconnect between what the human vestibular system (your sense of balance and spatial orientation) is telling the brain your body is doing vs the acceleration forces that your body is feeling vs what your eyes are telling you about what the body is doing.
Or in other words - differences in movement/locomotion your eyes are seeing in HMD vs what your other senses are experiencing.
Which probably would include latency as a subset.
To note, simulator sickness is not that new, we have been grappling with this for ~65 years since the first flight simulators. Despite the massive funding that the DoD has at it's fingertips, we have not found a cure for it OR there has not been a lot of work done to find a cure. However, it seems that more time in the simulator does help, thought it may then hinder actual flight performance. Also, the more experienced pilots had a higher occurrence of it. Again, it's not that well understood.
The vestibular system has a prodigious amount of control over your life. I've got hearing damage and occasionally get vertigo spells (I carry dramamine in my wallet for emergencies). Every once in a while my balance sense just begins to swing around wildly. Sounds kind of cute. It will put you on your ass for 24-48 hours with zero possibility of doing anything. Even drinking water is hard. Without drugs the nausea just becomes your entire world. You think about nothing but how much it sucks. You can't eat, move, sleep or speak.
So... yeah. Motion sickness is a very real thing. I think (based on zero evidence) that everybody has a different threshold but experiences the same effects past that. If you're lucky, your threshold is faster than you can move your head. If you aren't, chug dramamine and coffee and hope they invent something that turns off your sense of balance without making you delirious.
It's a pretty demanding latency test, and I can also see how it ties into balance etc (there are some similarities with a tilting boat deck, for example).
That said, I think simulator sickness and sickness from just playing on a big screen is complicated - maybe some will never enjoy it.
Personally I'm rather sensitive to motion sickness from most fps games - but I enjoy playing elite:dangerous with a vr headset.
Rollercoasters and smooth/fps style movement is rarely used - and even if it is, the interface is designed tomminimise the effect you mentioned - e.g. user is put into a virtual cockpit, which makes the brain feel it's the terrain that's moving and not the person.
Are you responding to someone? Because this doesn't really make sense to me. Can you clarify?
Maybe it's a due to a lifetime of playing video games (3d since i was 10), but even VR games with weird detached cameras or games not designed for VR which don't really respond to head movement are totally fine for me. There is a cognitive adjustment I make which is manageable and does not make me feel sick.
Having used gearVR with a scratched phone screen for a year and now real Oculus for a few months, I am fairly certain that my particular sickness comes from the strain of my eyes constantly focusing too close or trying to mentally blur obvious pixel artifacts or struggling to focus on fogged up parts of the lens which is causing me to experience a headache which eventually leads to nausea.
Because when VR/IRL movement are the same, there's no need to fool the body's systems.
The target audience for VR is everyone.
A lot of discomfort for relatively little (at least thus far) gain.
Aww. But I already had my scalpels out for DIY inner ear surgery. I just finished watching the YouTube howto and everything... :(
1) Don't move your POV in VR.
2) Train your brain interpret movement in VR as world movement not body movement. If you can maintain a sense that you're not actually moving just because your eyes are showing a lot of stuff moving around you, then your vestibular system and vision system will be in accordance and you won't have any motion sickness.
Children who use VR from a young age will have an easier time of this, but adults can learn it too.
People with really bad cases of motion sickness have problems playing games on a flat screen let alone in VR. I don't think we'll be able to do anything for them in the near future.
Even a rough version of the deformable mirror AR VRD described by researchers at UNC Chapel Hill [1] accomplishes 100 degrees FOV with accommodation.
They went further with the Pinlight achieving 110 in 2014 [2]
The technical limit according to our own work for VRD FOV is H: 200o, V: 140o (Combined). So either they're ignoring work in the field intentionally cause they don't want to do VRD or they don't know about it. My guess would be the former.
[1]http://telepresence.web.unc.edu/research/dynamic-focus-augme...
[2]http://www.cs.unc.edu/%7Emaimone/media/pinlights_siggraph_20...
edit: I find this whole thing extremely frustrating. Facebook could throw 2 billion dollars at VRD tech and actually get to a working stable consumer grade system if they wanted to - everything is there for it. Why aren't they?
I use 2B as an anchor based on the "official" oculus acquisition.
You can say that about any other multi-billion dollar company, let it be Google, Microsoft, Xerox, Cannon, anything.
The short answer is that VR is not their core business.
How is that not what they are planning as their core business?
1. Display resolution is still quite low, so really everything is blurry.
2. You will never be able to notice blurriness where you aren't focused anyway because you aren't looking there! Everything is always blurry in your peripheral vision.
3. Surely eye focus is a feedback system, like in cameras? I mean nobody has problems focusing on TVs because your eyes just magically change focal length until the image is sharp.
I am stereoblind so maybe it is a big problem for others.
Also it depends on the HMD screen size, at some point parts of the object's image for each eye falls off screen.
When I'm in VR, I notice it slightly, but it's easy to get used to. However, as soon as I get out, I find it very difficult to focus normally again, usually for about 4x the amount of time I was in. This can make it difficult to do anything that benefits from depth perception, such as driving, walking on uneven terrain, etc.
If this technique can improve that side effect, it would make quite a difference for me personally. No one else I've talked to about VR have this same experience, so it might only affect a subset of the population.
I was reading it specifically in the context of problems with displaying text in VR.
Paraphrasing: in real life, your body uses two inputs to intuit distance: your eyes' focal distance and your eyes' relative angle. Focal distance is fixed inside the headset, so your body loses one of its depth cues. I suspect that contributes to the overall sense that seeing things in VR can get tiring unusually quickly.
Full light field displays have to draw every ray, not just every location; that's two additional dimensions. Resolutions scale badly, and if you make any of the four dimensions lower resolution, you'll get ugly artifacts.
Microlenses aren't the limitation. Pixel density is, both in physical manufacturing side and when refresh / drawing.
I keep thinking there will be a significant reduction in complexity there. The intensity of a pixel in a hologram is essentially an integral over all surfaces visible from that point. So imagine a rather complex formula applied for each surface for each pixel. Then imagine holographic bounding boxes that compress complex geometry into a few holograms of what's inside. This would reduce the n^4 back down, but the resolution required for holograms is still very very high. But we could use fancy GPUs to evaluate the integrals.
Just hand-waving thinking here...
Items in infinity emit parallel rays, items very close to the eye emit rays with a large spread...
Any data compression method has to be compatible with the underlying display technology so you can bring the signal straight into the modulator. In addition, there's a wire problem: you can only bring so much signal into a light modulator because the signals interface through a surface, not a volume.
For these reasons and others, true holographic displays are simply not competitive at this time with stereoscope-like VR systems, which rely on tried-and-true, market-driven incoherent-light raster display technology.
People have been asking these kinds of "why not" questions about 3D and building various types of 3D display technology in garages and labs for almost two hundred years. With the exception of the discovery of optical holography, it's surprising how much is fundamentally unchanged.
This Facebook optic uses the same technique.
Or did I get that completely wrong?
I say this as a VR enthusiast who is optimistic about the industry.
You can't say it's solved until an HMD can forcefully move your viewport in-HMD without triggering nausea caused by the real body not moving.
Link to Paper: https://scontent.oculuscdn.com/v/t64.5771-25/11162698_189852...
Practical electronic 3D displays requires bandwidth reduction, both data bandwidth for transmission and optical bandwidth to create practical or lower-cost optical modulators. The goal is to use bandwidth reduction techniques that produce little or no visual artifacts. Some of the techniques used are the same as in 2D (spatial discretization, time multiplexing, compression), while others are unique to 3D (view discretization, limits on view angle, elimination of coherence).
Head-mounted displays are basically descendants of stereoscopes, the first 3D displays developed by Wheatstone in 1838. Wheatstone's amazing discovery was that you can throw a huge amount of information about the world away, provide just two images from two viewpoints, project them out to infinity in front of a viewer's two eyes using two lenses/light paths, and a vivid sense of 3D is evoked. That's an incredible amount of information reduction from real life.
In the traditional stereoscope, accommodation is thrown away, mostly because its really hard to recreate electro-mechanically, but also because we're generally fine with it. Accommodation isn't effective for distant objects (or for even larger depth ranges as we get older we lose our ability to accommodate), so we likely have neural circuitry to discount imperfect accommodation cues. One of the reasons we turn on bright lights when doing detailed work is to stop our eyes down and increase our depth of field, reducing the need for accommodation.
However, there have been perennial debates about the physiological impact of conflicting depth cues involving accommodation, and those debates are more interesting in VR where objects can be (virtually) very close to the viewer and the viewer can dynamically change their physical relationship with virtual objects.
Until you have a light modulator that can let you experiment with selectively modulating accommodation within a scene, you can't provide real data on how important accommodation (even approximate accommodation) is for a particular application. Can't wait to see the studies.
We did some similar focal plane manipulation in holographic video more than a decade ago, for related reasons (see Fig 7):
https://www.researchgate.net/publication/255603167_Reconfigu...
If just for this, it's a move in the right direction.
Is this technology compatible with foveal rendering?
During normal, outside-of-headset vision, we focus naturally and quickly on whatever we're looking at. We don't spend time with our eyes consciously defocused on subject matter in our foveal view. So anything that's out of focus will tend to be in our peripheral view.
So this is a peripheral technology. I think everyone's still looking for the killer additional tech that will make VR perfect -- but it's not about one magic tech bullet, it's about ecosystems slowly growing, and content getting better. (The headsets are better than people think.)
