> What are those 175 billion parameters?
> According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter can be thought of as a synapse in a human brain ...
> They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.
So his estimate is now off by a factor of about 100? How long will it take for the next couple of 100-fold increases in parameter count?
