Part of the reason I worked on this in my spare time (my real job at Google is something else) was to get it integrated in libmv (http://code.google.com/p/libmv) and subsequently integrated into Blender. This opens the door to many sophisticated tracking features, like autocalibration, rolling shutter, multi-shot solves, tripod solves, planar tracking, and more.
EDIT: Changed link title to make it more provocative
If you're snapping track data to roads, how do you deal with areas where you have no existing data? Do you take into account aerial imagery?
</openstreetmap-nerd jealous="true">
I took some notes during the StreetView Q&A on reddit two years ago: http://blog.est.im/archives/554
I was looking forward to integration of motion tracking into Blender and had a small play with the feature when released. Now I just need to make the time to play some more...
Note that Blender's next open movie, http://mango.blender.org, is live-action done with Blender's tracking tools. Ceres is going to be used for that as well.
This is extremely cool. The documentation seems to be a WIP on this, can anyone comment on what kind of cost functions are supported by automatic differentiation?
EDIT: it would be also interesting to see how this compares to Theano [1], which also does symbolic differentiation and can JIT to native code and GPU. It is a largely more generic framework, but I'm not sure how well it can handle large sparse problems.
What happens if I try to use an if-then or a for loop depending on one of the variables?
I think it can be used as a backend in Ceres to push computations to the GPU, but as it stands it is a different category of software than Ceres.
While we do support numeric differentiation, we don't suggest you use it. What we have is automatic differentiation, which is a technique to take exact derivatives without resorting to symbolic differentation.
Check out he jet.h file which is the core implementation of this:
http://code.google.com/p/ceres-solver/source/browse/include/...
The header comment has a nice description of autodiff.
In summary, Ceres supports three ways to compute derivatives:
(1) Automatic differentiation (easiest, fastest, most accurate)
(2) Numeric differentiation (easy, but worse convergence and hazardous)
(3) User-supplied jacobian (use a pen and paper then implement the jacobian for your residual manually)
It's really surprising how broadly applicable non-linear least squares is.
For example, in the subproblem of only one-way streets that we can assume to be composed of one or more segments of straight lines per street, we take a simple street-function $\phi_i(x,y)$ corresponding to street i to be the norm of the projection of <x,y> onto that street. Furthermore we also add into the system some "smoothing-function" to ensure that the overall shape of the final path is doable, for example constraining the distance between successive points. Next, we solve the argmin of the norm equation for each point so that each point is now moved to some linear combination of the streets, and truncate all but the most significant street basis, and rerun until we get to some acceptable tolerance.
Maybe?
Its a bit like Google being the biggest supporter of Firefox, even though they are also a competitor (Chrome.) Though Google supporting OSM is more like Microsoft supporting OpenOffice, since Google has such a large monopoly in maps right now.
