I did this at the time. I downloaded his Wii app that shows the little red/white discs, and bought a battery powered Wii-bar thing that had the IR leds in it and strapped it to my head.
Can confirm, the 3D effect is 100% spot on, even the smallest move of your head caused the display to adjust perspective and your brain is totally fooled into thinking the Wii display is a just a window into a 3D scene.
tbh, it was the only cool thing I actually did with my Wii!
Apple later added the effect to the homescreen in iOS 7, but much more subtle.
This sounds way too interesting, so I'll be doing some digging, apparently. Any idea if this would work effectively on a large projector screen?
Funny you should ask that! My set up IS a large projector screen (110"), and yes it does work. In fact the size enhances the experience. :D
Your brain takes a lot more cues for depth than stereoscopic vision. The way the perspective changes as you move your head around is sufficient even for a flat display. It’s uncanny.
Only works for one person though. If you stand at an off angle to the sensors, there’s no effect at all.
That it's not based on head tracking, and in a product you'll be able to reasonably buy.
Not sure if they ever ran Doom on it though.
From the comments, here, it sounds like it might. There's some interesting applications, here. Thinking back to last night, it would have been really nice if when I was looking at the split-screen in my driving game with my kids, my own corner had that kind of depth to it. Hell, driving/flying games, in general, suffer from realism issues due to not being able to faithfully produce the effect of looking out a window.
Doing this on a larger screen (and throw in a transparent LCD after you work out how to bend all of this into that, assuming it can be done) and you could have a light-switch that changes the scenery from "outside" to "outside somewhere else"[0].
[0] I remember seeing that in Back to the Future II as a kid and thinking it'd be really nice to have on a cold, gloomy, February day in Michigan.
- Split a 4k signal into 45 view planes means the effective resolution sucks, bad.
- Likewise, if your content is complicated at all you need a monster GPU because you are rendering your scenes 45 times per frame, meaning 45 times the draw calls.
- Field of view is very limited
- The amount of z-depth you can put content in without major blurring is much lower than even this video would indicate.
- You need to design your scene so important content never reaches the edges or things look yuck.
- Some patterns lead to artifacts like moire, and it can be hard to predict. Your content needs to work around this.
- They are (understandably) very expensive.
All that said, these things are cool and would be a perfect fit for some use-cases. After the prototype I built was green lit for full production, the decision was made to ditch these displays for all the reasons above.
1) Isn't this just based on "naive" implementations? Surely there are optimizations to not have to render all 45 view planes at all times similar to how 3d rendering doesn't render scenes that are not visible until the character moves to change perspective?
2) The difficulty with layout and design doesn't seem that different from any 3d or game design challenges to make sure the person can't see the wizardry behind the curtain. Is it?
I guess what i'm thinking is if you take an Introduction to Computer Graphics course, you can learn all the math, all the concepts, and all the abstract ideas about building 3D worlds. Then you try to implement them and you immediately realize that none of the ideas are performant enough to actually put into practice in a video game without MAJOR optimizations that on the surface seem 1/ Hacky, 2/ Extraordinarily difficult. We've been doing a range of them for 30 years. But some are only TODAY entering the mainstream despite being the simplest possible concept to explain to a student (ray tracing).
Is it not the same situation with these kinds of displays (and other VR/AR)? We're in the super early stages and we need the John Carmack equivalents to identify all the super insanely clever optimizations to squeeze something practical out of them?
It would be better if we could just have 3x 2D projectors that somehow interfere in a medium to display, but I don't know of a way to accomplish this. Not to mention none of this allows you to be inside the displayed content as VR does.
Maybe you could add head tracking to prune some angles, but if you were going to do head tracking, other comments have reports of good results without a lenticular setup. You'd probably lose out on the multiple viewers aspect.
I would think naively that there's some amount of work that could be shared between so many renders from nearly the same viewpoint. But that probably requires driver changes to make it available, I did find an article from NVidia about rendering four viewpoints in one pass, which might be applicable and help somewhat: https://developer.nvidia.com/blog/turing-multi-view-renderin...
For 1)
- Their SDK implements this rendering strategy, so either they haven't been able to justify the effort for additional optimization, or (much more likely) there isn't a tenable general purpose solution that doesn't fall apart horribly in many circumstances. This latter point is true of many optimizations in games, they work because you can very narrowly scope them to what you are doing.
- Frustum and occlusion culling are used, like you mention, to avoid rendering meshes that are off-screen. This isn't a panacea though, there can be significant CPU cost to doing so to the point where it can be more performant to disable it. Case in point, I used Umbra occlusion culling in my VR game in only a few scenes because my CPU budget for 90 FPS was so small that the PS4 couldn't keep up.
- Those 45 planes are a series of slices that are very closely projected in space, so the odds that there isn't any work to do is almost nil. Even more true when you consider dynamic scene elements, such as lighting and shadow casting.
For 2)
- All of the challenges present for 3D content creation are unchanged with this device, it just adds more on top (limited FOV, depth blur, and fringe blur). You can't wizard away seeing color separation at the border of the screen, it's just fundamentally the hardware's limitations.
Another example, the PSVR headset's OLED panels cause a purple blur to happen on edges that have too much light/dark contrast. The solution? Call an API and change a floating point "base brightness" level for the headset, or change your lighting -- for each level (this is Sony's recommendation!). Entirely game and scene dependent, and not something that an automatic solution could accomplish.
- Some of the "magic" techniques invented for VR performance may have the possibility to be applicable (skipped frame re-projection, foveated rendering), but those are based on the math between two camera projections not 45.
- Any improvements that could be created by a Carmack-level person are going to be market driven. The population of people that fit in that category would have the opportunity to work on whatever they want, and VR was compelling to many of them because of how damned cool it is (was?). By comparison, this tech is a novelty that a regular consumer will probably never see.
But I don't understand how it works. Seems like... head/hand tracking? But then why the weird-looking screen?
A bit disappointed, at first I thought it was something similar to those cards we had when we were kids, that change image depending on which angle you look at them (forgot the name).
Edit: seems like it's a bit of both: https://docs.lookingglassfactory.com/KeyConcepts/how-it-work...
Edit2: Okay so it does work like the cards we had when we were kids. The screen shows 45 different views of the scene at once. I'm back at being non-disappointed.
¯\_(ツ)_/¯
When I looked at the animations/video, I thought it was really slick and as I read more about the tech it appeared to be as I expected, but I'm not sure if I should be disappointed by that.
I think it was quite intelligent for the company to offer a smaller, relatively inexpensive ($249 early-bird isn't bad) display because I fully expect it'll be impossible to properly evaluate the quality of the tech without actually seeing it live. The few times I've played with some of the more exotic screens (even oddball LCD screens like the promising, but brief IGZO LCD panels) are very difficult to evaluate. The videos are usually far from what it looks like in-person (and I'd wager about 75% of the time, the video makes displays look worse than it is).
I'm curious if you can speak to the down-sides of doing things this way. Were this screen a "real holographic display" but with the similar constraints[1], would it be substantially better? From the videos, I get the impression that this screen can only display 3D within the bounds of the "box", so if a real holographic display behaved similarly, what could it do that an autostereoscopic screen cannot?
I'm curious because I was left behind during the 3D craze (that disappeared, as I predicted, a few years later). I can handle about 30 minutes of 3D-glasses before I start getting the early symptoms of Migraine. I was hopeful when the TVs came out that I might be able to watch Avatar[2], finally, but I tried various sets with different types of 3D glasses (active/passive, I recall?) and they felt more uncomfortable than at the theaters[3]. I've never been diagnosed with lazy eyes/other eye problems, but I feel like my eyes go cross-eyed with the glasses on.
I'd have to see this, physically, to be comfortable with buying it. At the price-point of their smaller option, that's getting pretty close, though. If they had a very convenient/complete return policy, I'd probably check it out.
[0] That's not to diminish the value of the comment, I had only assumed it was some form of "Glasses-free 3D", but know nothing about the tech or how it works.
[1] As in, the dream is a "projector like" screen where a hologram could just be displayed in an arbitrary location in a room, is out.
[2] I have not seen it, yet. When it had finally died down in theatres, I had read that the story was "not very interesting/creative" and "kind of dumb". This was supported by the fact that everyone I know who went to see it told me nothing about what the movie was about and not one of them mentioned anything they liked about the "story". I think the best summary I received was from a close friend who said "I left the theater and 'the world' seemed a little less real" and I was interested to see the result of this camera that was invented for the purpose of filming that movie, but I haven't seen a second of it, yet.
[3] I thought it was normal to feel "off". I call it "almost dizzy" because I don't feel off balance, it just feels like it takes an amazing amount of effort to pinpoint objects with my eyes -- screen or otherwise -- with any kind of 3D glasses on. I wear a very low prescription pair of glasses (not required for driving, I'm nearly 20/20) and other than really cheap sunglasses, I generally have no difficulties otherwise.
1. Your brain recreates the 3D environment from physical objects and is "trained" to certain distances and movement, etc. When you use 3D glasses, either movie or VR, the "stereo" effects are anything but natural. There's a lot of stuff flying around at "non-natural" distances, especially near you. The effects are exagerated otherwise at some point you will "stop noticing it's 3d unless you pay attention". The exagerations are compensated by your brain and also causes eyestrain. Look at a pen, closer, closer, at some point you see 2 pens, now do that back and forth for 30 minutes, You will have the same pain.
2. Perceived movement vs. internal ear and balancing your body. VR is awesome at screwing everything up. I love VR but can't stand it for a long period because of that. Your brain expect you falling, or this thing comming at you to hit, you walking should have some motor feedback, driving too. Nope. No G's, no wind, no nothing. To put it in simple terms... brain tilts
you can look up a lot of stuff about this on the internet with VR sickness, motion or 3D sickness... lots of articles, i.e. https://en.wikipedia.org/wiki/Virtual_reality_sickness
I had problems playing Quake on a 2D screen. It went out with intense exposure. I would do 30 minutes, get sick, start the next day, eventually I've managed to pull out 12 hours in a row. Life was fun and simple back then ;). I don't have that luxury anymore for VR but I'm sure it would be the same process to desensitise through gradual increased exposure.
So far “holographic” seems to describe the experience of the viewer more than a technical spec.
[0] Garage/shed/lab
[1] Not implying this would come even close, especially not being a "true hologram", but it'd still be the closest we could probably get with current tech outside of headgear.
Then I saw a more recent version at a coffee shop in Providence. They're up to legit resolution now, and as a poster said, the light is split so all you're doing is moving your head. No head tracking/whatever.
It's one of those things that's tough to "get" without seeing yourself. Feels like a definitive piece of the future. I joined their Kickstarter awhile back as well.
Go Looking Glass crew!
Edit: the video linked here by teekert does it correctly
And here's a Linus Tech Tips video of it: https://www.youtube.com/watch?v=-EA2FQXs4dw
They have some good footage (though obviously it's setup as a hype reel too).
Edit: Skip to follow for "beauty shots": 4:45 and 8:50. 6:12 for funky shot of the 'flattened' image to give you a sense of what 'trick' they're playing
