...which fits into a size of less than 700Mb compressed. Some of the most exciting stories I've read recently for machine learning are cases where learning is re-used between different problems. Strip off a few layers, do minimal re-training and it learns a new problem, quickly. In the next decade, I can easily see some unanticipated techniques blowing the lid off this field.

It indeed strikes me as particularly domain-narrow when I hear neuro or ML scientists claim as self-evident that "humans can learn new stuff with just a few examples!.." when the hardware upon which said learning takes place has been exposed to such 'examples' likely trillions of times over billions of years before — encoded as DNA and whatever else runs the 'make' command on us.

The usual corollary (that ML should "therefore" be able to learn with a few examples) may only apply, as I see it, if we somehow encode previous "learning" about the problem in very the structure (architecture, hardware, design) of the model itself.

It's really intuition based on 'natural' evolution, but I think you don't get to train much "intelligence" in 1 generation of being, however complex your being might be (or else humans would be rising exponentially in intelligence every generation by now, and think of what that means to the symmetrical assumption about silicon-based intelligence).