undefined | Better HN

0 pointsjoefourier3mo ago0 comments

That depends what kind of ASIC you’re talking about. Cerebras can run models like GLM 4.7 with 355B parameters.

0 comments

2 comments · 1 top-level

cubefox3mo ago· 1 in thread

Cerebras just uses SRAM instead of DRAM. An ASIC would instead hardwire the neural network.

Surely it's more of a spectrum? From a CPU, to a TPU, to a chip that hardwires softmax attention but lets you store arbitrary weights, to one that hardwires the weights directly.

j / k navigate · click thread line to collapse