undefined | Better HN

0 pointsblackbear_5y ago0 comments

Just because many more humans spent many more years and many more $$$ building GPT-3 for your convenience.

0 comments

3 comments · 1 top-level

fastball5y ago· 2 in thread

Right, but GPT-3 can be used generally. That's the difference. It scales because you don't need to build an entirely new model for each different use case.

You just change the prelude and use it for something new.

sillysaurusx5y ago

It sounds like a big deal. What a tempting idea. And a colleague was mildly annoyed with me for how unimpressed I seemed.

But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.

I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).

fastball5y ago

I think the onus is on you to prove that the use cases are shallow and limited. I've seen GPT-3 already being used for diverse and interesting ideas that would not have occurred to me personally.

However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.

Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.

3 more replies

j / k navigate · click thread line to collapse