Lytro Lightfield Gallery (opens in new tab)

It's extremely interesting, for two groups of reasons: (1) creative possibilities (playing with depth of field is my favorite part of photography); (2) market applications.

Regarding (2), if I understand the paper properly, this should allow for a massive increase in lens quality while also allowing for lenses to be much smaller. Both are worth tons of money... together, it's massive. As a guy who carries around a $1900 lens that weighs 2.5 lbs, because of the creative options it gives me that no other lens can, this appeals to me greatly!

Refocusing the picture after the fact isn't just about being able to focus "right" after the fact, so it's not fair to just compare with "focusing right the first time". With normal cameras, when you focus, you lose information, and it is simply not possible to do what they do in their demos, namely, to capture a continuous range of fields of view. (With several cameras or more than one lens, you can capture multiple discrete fields of view, but not a continuous range like this.) This is only possible because they're capturing 3-dimensional information about where each object is.

Granted their demo isn't impressive, but they're underutilizing their technology, and honestly I can't think of a better demo either, but don't be misled. This light field camera is capturing far more information. Meaningful information. I wonder if it's possible to like, create 3D models of objects in these images? That would probably be more "computational camerawork". What's impressive that that could be done after the fact.

How about macro photography. When you get really close to something your depth of field shrinks. This lead to the invention of focus stacking. If instead of having to take 4 pictures, you can now take only 1, you can capture incredible things.

https://secure.wikimedia.org/wikipedia/en/wiki/Focus_stackin...

Mark Levoy (and his group) do all these things. However, if one wants to track through crowds one needs a much bigger aperture. So one would combine many small cameras into an array as opposed to a microlens array infront of the sensor.

Hacked some python script together for parsing this and generated some HTML file with the data out of it. Still not sure when to change the frame, the whole thing is of by some pixels.

Surely those are just standard depth-maps as used every day in VFX for compositing? Each pixel has a z-depth as to how far it is from the camera, and using standard compositing software (Nuke) you can blur just a slice (depth-wise) of the image based on the z ranges?

Presumably the data format is based on their light field viewer: http://lightfield.stanford.edu/aperture.swf?lightfield=data/...

(note this downloads 12 Mb of data)

You can look at the source: http://lightfield.stanford.edu/aperture.html

Ren Ng, a former student of Marc Levoy's (at Stanford), started the company. Ng was widely believed to be one of the rising superstars in computer graphics/computational imaging when he decided to leave academia and start Refocus Imaging.

The company seems to be doing well, and recently changed names to Lytro in order to not be pigeonholed into refocusing applications only.

You could hack up something very roughly like it with Kinect but it wouldn't work as well, it'd be a neat hack but not actually usable for anything serious.

Kinect's image capture is very low resolution (even by today's cellphone camera standards), it doesn't give you depth information at a per-pixel resolution and even ignoring those issues in addition to depth information you also need a source image which has critically sharp focus across the entire viewing range (you can't selectively focus in software that which was captured out of focus with a standard digital sensor), which means using a very small aperture (large F-stop value). So it'll be very difficult to capture anything but still-life images because the small aperture means a long exposure time, and thus motion blur if anything moves. Granted this is already less of an issue with Kinect because the sensor in it is so tiny that getting out-of-focus areas is not that much of a concern, but the cost of that is that the image resolution is also atrocious.

Once you get up to usable sensor resolutions, if you're already limited to taking long exposures of still-life images on a tripod, you might as well skip the IR depth perception and just take a series of wider aperture pictures at different focus plane levels, focus stack the results and preprocess the image series for blur levels to work out the relative depths of the in-focus bits of each source image. At least doing it that way you can use a DSLR to get quality photos.

Neither of these is a true replacement for what they are doing here, though.

For the Kinect to work you need a surface where the infrared spot pattern is imaged. For most of the things we look at the Kinect will be sufficient.

However one captures a lot more information with the light field camera. For example transparent things like smoke, fog, glass and things with weird optical properties like polished steel or the Tiger eye mineral with Chatoyance will be captured by this camera.

This gives the photographer tremendously more artistic space. Just imagine photographing a close up of an eye with the Kinect technology.

One can argue that the light field camera will maintain a quality in the image one could never achieve with Kinect based systems (without a lot of photoshopping).

The idea has been around since the turn of the century.Lippmann proposed it [1]. But the first guy to BUILD a light field camera was a Russian in 1911 [2]. He built a light field camera AND display using a copper plate drilled with 1200 pinholes and reconstructed the light field of a lamp.

BTW, your comment illustrates that much of "computational photography" is just re-application of tricky imaging tech from other fields. Nothing wrong with that, but something to keep in mind.

[1]http://www.tgeorgiev.net/Lippmann/index.html [2]http://www.futurepicture.org/?p=34

My coworker went to GoogleI/O and got a Galaxy Tab 10.1. With an Android app that meets that description. I cannot take another loom or get it's name over the weekend, though.

It also had a feature that automatically saved the grams of a video that had a face wit a smile.

Tried it just now and it is merely a "technology demo" rather than a proper app. Having played with it for 20 minutes I wasn't able to make a single image that was worth saving, but still it was a buck well spent.

This wouldn't work with the picture of the bird flying http://lytro.com/gallery/content/lytro_50_00087.php

There is a nice talk by Levoy. He is trying to build a motion camera:

Why is Sports Photography Hard? (and what we can do about it)

http://research.microsoft.com/apps/video/dl.aspx?id=146906

So for sports photography it seems very useful. Replacing the green screen doesn't seem to make sense.

For macroscopic objects depth reconstruction with two cameras like in the Kinect seems the better alternative.

Object excemption seems to be a nice idea. I can't remember having read anything about this. In principle it should be possible to recover an unsectioned stack of images.

One can then use an iterative algorithm to subtract in-focus information from on slice of the stack from each of the other slices and end up with a deconvolved image of sectioned images.

Then one could delete one object in the stack and recalculate a superposition of blurred sectioned images to recover a reconstruction representing the object without the image.

This is quite complicated. Just imagine to remove a wine glass from a scene. One needs to delete all the rays that went through the wine glass and bend them such as though the wine glass wasn't there.

One can argue that polarization and absorbtion effects will be very hard or even impossible to handle correctly.

Certainly light fields contain A LOT of potential.

This is pretty much the same technology that Adobe demoed back then. Todor Georgeiv and Chintan Intwala also refer to Levoy et al. in their papers, e.g. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104...