The development of this chip shows that it doesn't (and shouldn't!) matter to the ML teams at Meta how 'fast ML is evolving.'
Indeed what it demonstrates is that a huge, global, trillion-dollar business has operationalized an existing ML technology to the extent that they can invest into, and deploy, customized hardware for solving a business problem.
How ML "evolves" is irrelevant. They have a system which solves their problem, and they're investing in it.
You've gotta learn to walk before you can run
And building out specialized hardware does lock you in to a certain extent. Want to use more than 128GB of memory? Too bad, your $10B chip doesn’t support that.
Which is probably why Meta is also buying the biggest Nvidia datacenter cards by the shipload. There is no need to run inference for a small model - say for a text-ad recommendation system - on an H100 with attendant electricity and cooling costs.
You don’t always need a Ferrari to go to the store
It’s custom silicon designed for a specific, known workload. It’s not designed to be a general purpose part or to be future proofed for unknown future applications.
When a new application comes along with new requirements, the teams will use their experience to create a new chip targeting that new application.
That’s the great part about custom silicon: You’re not hitting general specs for general applications that you may not even know about yet. You’re building one very specific thing to do a very specific job and do it very well.
At Facebook's scale the spherical cow raw performance stats don't matter nearly as much as real world workloads per ops dollar. They can also repurpose their GPUs to other workloads and let their custom chips handle the boring baseline stuff.
