Edited: See (1)for some related ideas: A Safe Approximation for Kolmogorov Complexity
(1) https://link.springer.com/chapter/10.1007/978-3-319-11662-4_...
This is studied in so-called algorithmic rate-distortion theory:
Rooij, S. de, & Vitanyi, P. (2012). Approximating Rate-Distortion Graphs of Individual Data: Experiments in Lossy Compression and Denoising. IEEE Transactions on Computers, 61(3), 395–407. https://doi.org/10.1109/TC.2011.25
Vereshchagin, N., & Vitányi, P. (2006). On Algorithmic Rate-Distortion Function. Information Theory, 2006 IEEE International Symposium On, 798–802.
The "common wisdom" of "too many parameters will make you overfit" is most definitely not that important for the way modern NN training works.
It walks side-by-side with compression (and pigeon problems). Using Kolmogorov to improve ML in those physical examples means that the solution will be better to the specific case, not that there'll come a one-in-all solution to any kind of clothes animation.
If you have N+1 things and N pigeonholes to put them in, at least one pigeonhole must have more than 1 thing!
That quote is in reference to tasks such as physics simulation. There is an incredible GIF in the OP which shows a digital mannequin being manipulated, with its dress flowing in a hyper-realistic manner due to ML physics simulation. It would be uncanny to see that type of simulation combined with AR.
I'm curious to what extent ML physics simulation may be beneficial for self-driving cars. Generally, we as drivers know the physical properties of objects that we can collide with. Cars don't have that understanding, so they might "think" that colliding with a large paper bag is unacceptable. Stopping suddenly because of that paper bag may be fatal.
I know some papers that try to improve physics simulations with deep learning and I think it’s definitely possible, not sure though if it can really improve most physics-based simulations.
Physical models are already highly condensed and have the advantage of being interpretable, deep learning has a long way to go before it could be used as a replacement, IMO.
I don't know enough to say how accurate Hutter's explanation is but it made sense to me.
