undefined | Better HN

0 pointsconvolvatron3mo ago0 comments

exactly. in HPC we all understood that it was a tradeoff between money and time, and that the curve was exponential. if you wanted to race ahead of todays capabilities, you could, but you couldn't go very far without burning alot of cash.

because of the investment story about being first and building a moat, we have companies torching 100s of billions of dollars to see who can climb that exponential the furthest.

we have so much work to do, in infrastructure, and distributed computation models, and programmability, quantization, and information theory...just relax a little. you dont have to compete with OpenAI. OpenAI is just a giant waste of money. take your incremental gains and invest in research and I assure you we can get there without directing our entire economic output into buying the latest highest margin parts from Nvidia only to use them at 30%, if you're being generous.

0 comments

bborud3mo ago

This kind of reminds me of an old paper: https://arxiv.org/abs/astro-ph/9912202

Of course, we are no longer in the clock speed doubling era, but computer does get faster still.

convolvatronOP3mo ago

It's true that the progress on clock speeds has slowed. Now we have to address the parallelism problem in order to keep moving forward. And we haven't done a very good job. Progress on that front will get us back on the acceleration curve. Saedfly, the current framing of 'who buys the most hardware', while I providing a nice marketing story, isn't netting us that much progress except what Nvidia spends internally.

j / k navigate · click thread line to collapse

0 comments

bborud3mo ago

This kind of reminds me of an old paper: https://arxiv.org/abs/astro-ph/9912202

Of course, we are no longer in the clock speed doubling era, but computer does get faster still.

convolvatronOP3mo ago

j / k navigate · click thread line to collapse