undefined | Better HN

0 pointsbaryphonic2y ago0 comments

GPT4 is also much more expensive to use, even at the probably-running-at-a-loss pricing. Accuracy per dollar and/or joule doesn't seem to be increasing too much.

0 comments

brookst2y ago

Sure, if we’re changing the complaint from “accuracy won’t get better” to “efficiency won’t get better.”

But I don’t believe that either. Hardware design and manufacturing cycles are long and we’re just now starting to see transformer-optimized HW. Accuracy per dollar and per joule will improve by at least an order of magnitude, maybe two.

But then we can move on to percent of human tasks LLMs are suitable replacements for, or the ethics of increasing automation.

baryphonicOP2y ago

I was critiquing accuracy of ChatGPT, which has a certain cost. "Just use GPT4, 5, ..." was the solution proposed in response, but that ain't free. In fact, it's one to two orders of magnitude more expensive.[0] I'm also pessimistic on custom hardware being that much of a help. I read about Cerberus's giant wafer chip in 2019[1] and even saw it at a conference in early 2020 pre-pandemic, and I have yet to see that make orders of magnitude worth of improvement. The attention mechanism simply requires huge amounts of multiply-adds, especially at GPT-scale. That costs energy and cash no matter how much custom hardware you throw at it.

[0]https://openai.com/pricing

[1]https://www.wired.com/story/cerebras-chip-cluster-neural-net...

j / k navigate · click thread line to collapse