In this case, it’s fair to say the machine, by analyzing pixels, can’t figure out perspective very well. The human can do that just fine, given an interface mechanism.
The machine is good at detecting edges and seeing similarity between pixels. Given hints from the human that ‘this point is within an object’ and here is the perspective, the machine can infer the limits of the object based on edges/colors and project it into 3 dimensions. Amazing.
This is just in case you want to throw a few upvotes their way for being first. This also illustrates that late night (PDT/UTC -8) posts don't get a whole lot of votes and proper timing is crucial to getting lots of votes.
Personally, I'm just glad to see this video finally getting traction. It really is such a cool demo. It even stands out in the field of consistently high-quality SIGGRAPH demos. Can't wait to read the paper!
It's weird that it has received quite a few votes each time and never made it to the front page. Was it a timing issue (late night, early morning, non-American hours) or is YouTube "weighted down" somehow?
This is indeed magic. I'm so happy to live in this age, and be part of the "Sorcerers' Guild".
Also, with the shiny objects, could you specify the material properties and have it "back out" the reflection such that the reflection was recomputed as you moved the shape around?
Forget the Photoshop stuff, this needs to be integrated with 3D printing immediately.
Spit out a design file into Tinkercad[1] for some minor adjustments and BAM, you've made a printable 3D model.
This technology is awesome. If it's as user friendly as they make it looks, I could see a lot of application for that!
For example, I have only tried my hand at 3d modelling once or twice (and sucked at it enough to give up), but just watching this I feel like I could model vases and lamp posts with a bit of practice.
These guys/girls know what they're doing.
Indeed, and it's very impressive work.
It makes sense that this is the case, because this system is doing edge detection with fairly strict constraints: the edges must match the outline of a fairly simple shape which you roughly know the size and orientation of. That seems like it's inherently going to yield better results than completely-unconstrained edge-detection as in photoshop....
I wonder if it's just a coincidence, or whether the mega-bucketloads of money the film industry throws at CGI are a major factor in funding related research even in academia?
Sure, understood.
The thing is, I imagine film VFX guys are already doing this kind of task—making 3D versions of real objects from the movie and doing CGI additions from them—and tools like this (with, as you say, refinements) could be a great help in speeding up that process...
Edit: I am aware that Photoshop has some of this available. I've not played with it so I don't know how they compare.
The impressive thing here, imho, is the seemingly effortless and seamless transition and replacement. The background is fixed and the surface texture is stretched in what seems like real time.
I vote for this to be used with 3D printer
however, it seems strange in the first example how mountain ranges appear where none were before... how did the algos know to put it there?
