undefined | Better HN

0 pointsguax22h ago0 comments

I think that for coding we're past the plateau issue. The frontier models of today are good enough and very valuable. The expensiveness in running them will eventually be solved by cheaper faster hardware.

I do hope that a day will come where you can buy the nvidia spark thingy for 5k that can run the equivalent of Opus 4.6 or 4.5 locally and that would be a massive thing.

0 comments

3 comments · 1 top-level

johnvanommen22h ago· 2 in thread

> The expensiveness in running them will eventually be solved by cheaper faster hardware.

How?

* Moores Law is almost over. The 5090 improves over the 4090 mostly because of quant improvements.

* even if the hardware improves, there’s a huge incentive to slow roll the next generation. Nobody wants to end up like Sun Microsystems. Sun’s used hardware was faster than its new hardware, once you considered price. Sun ended up competing with its own used equipment.

The most obvious place for improvement is RAM, network and storage.

If someone can bring more RAM onto the market, that will unstick things.

Legend244021h ago

GPUs are not really the ideal architecture for running neural networks; they are heavily bottlenecked by memory bandwidth and struggle to keep all their tensor cores supplied with data.

There is significant room to make more specialized neural network accelerators with new compute-in-memory architectures.

If the brain can run 86 billion neurons on 30W it must be possible.

akomtu17h ago

Our brains run 86 billion neurons the same way a waterfall runs a fluid simulation with N quadrillion particles.

1 more reply

j / k navigate · click thread line to collapse