This is a good article on the topic: https://arxiv.org/abs/1902.03477
Recommend work & talks by Anna Gilbert for anyone interested. Entertaining & good at distilling technical content. Here is her most recent one, but there are other good ones on youtube. https://www.youtube.com/watch?v=Sb1ZhtsZjyM
Not to nitpick but that article is a year old and the field is moving at lightspeed
All in all, nobody really has a clue on how to do meta-learning right (or I am not aware of their work). There is progress being made on benchmarks, but some argue that progress is not really tackling the real issue at hand, i.e. learning to learn. Moreover, the current common benchmarks are not really good at untangling the progress in deep meta-learning from the progress in deep learning in general.
Shortcut Learning in Deep Neural Networks
Adults do (i.e. the agents pretrained holistic model of its entire observed physical context). By reducing the phenomenon to the single observation, you're conveniently ignoring the early childhood phases spent exploring shapes/3d-geometry that enable this very ability of inference. this isn't fair, because regarding humans, the line between training-phase and trained model is very blurry, whereas a statistical model is trained when the weights are set and done.
Brute forcing through 2d-projections of 3d-objects (further denormalized through camera-artifacts etc.) until something sticks in a convoluted (heh) composition of arbitrarily initialized set of nodes and connections is obviously far different from the physical exploration kids do. Comparing the models resulting from the latter with the former is, in a word, absurd.
Through exploration, humans develop a model of physics itself, from which the nature of cupness can be inferred (which is, in fact, the magic term).
Deep learning alone won't get us there, but it'll probably give us the components that enable us to simulate this intricate process happening in kids brains.
In fact, I'm pretty sure that that's what a lot of the smart people researching general intelligence are working on (because that's what I would do, excuse my hybris).
I think what I was looking at was the result that has been often observed, that progress in AI research roughly tracks with hardware developments. Looking at AlphaGo to AlphaZero to MuZero. Training time for self-play increases. But parallelism in the tensor units of the hardware is an order of magnitude faster. It's great for problem domains like autonomous vehicles, contactless payments in retail stores and fraud detection in the data center. But what about generalizability? What about the black box communicating how it has learned? Will it be suitable for next-gen applications like robots designed to assist humans in space expansion?
I attended an event in NYC around the creative use of AI by a new breed of emerging artists like Mario Kliegmann from Germany. ArtBreeder can train a GAN on a single input sample and generate paintings in the style of Fragonard or Picasso or Rothko. And someone made a remark along the lines of: "if this had existed in the 1960s, we wouldn't have need Warhol to invent Pop Art!". But in reality, Andy Warhol experimented with a wide variety of media and techniques. From film to "oxidation art". And it struck me that was the truly creative part of the process. One that arises from a place other than rational optimization on a single task or even multiple known tasks.
Well, that's what partly machine learning already does, right? :)
Did not get this part. I have limited sample of two kids, but I would say it takes at least a year before humans understand "cupness"
But, in general, deep learning requires far more examples to train image recognition and even then it's relatively fragile. (Not that humans can't be fooled but having models of the world in our brains help a lot. No, that's probably not a flying pig even though it looks like one.)
What new things will I be able to do after this course?
(in a practical sense, the technical description on the course page I can read myself)
I took a look at the course outline, and except for AutoML, it appears to be a one stop shop for learning multi-task and meta-learning. I just bookmarked the lectures on youtube.
