Multiple years have passed since then. GPT spark got me excited again, but somehow that seems to have faded right back into obscurity.
Can somebody explain why there’s so little apparent progress here despite the theoretically massive advantage? Can I still expect this to happen eventually?
In the future, they plan hybrid implementations, to be able to serve large models better, e.g.
"AWS. We signed a binding term sheet with Amazon Web Services for AWS to become the first hyperscaler to deploy Cerebras systems in its data centers. Deployment in AWS data centers will require us to meet strict standards for performance, scale, and reliability.Pursuant to the term sheet, we will create a co-designed, disaggregated inference-serving solution that will integrate AWS Trainium3 chips with Cerebras CS-3 systems, connected via high-bandwidth networking, to partition inference workloads across Trainium3 and CS-3. Each system will perform the type of computation at which it most excels. The approach is expected to deliver 5 times more token throughput in the same hardware footprint, at up to 15 times faster speeds compared to leading GPU-based solutions as benchmarked on leading open-source models."
So, the price makes it necessarily niche to some specific use-cases like HFT or intelligent duplex voice assistants, I'm still semi-bullish personally.
In practice they are also not very flexible when compared to gpus.
I think wafer scale could improve performance of models and has some applications, but from a manufacturing perspective this approach seems cursed. Defect density is irrelevant when your target is an actual barn door. You can make the system resilient to defects, however the tradeoff is that you have to hedge for defects being anywhere. With chiplets, you accept that some units of space will be completely unusable. The trade off is that others are much higher performance because we don't have to spend any space or time on redundancy.
