> And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic.

> But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall.

You're just talking about layers of abstraction. GPT-3 works with chunks of 3 letters. It can now.

Combine chunks to form valid words. Combine words to form syntactic sentences. Combine words to form semantic sentences. Combine sentences to form consistent paragraphs. Combine paragraphs to form trains of thought.

It can't combine trains of thought to produce a consistent point.

But it's already working successfully at 5 or 6 levels of abstraction up. I don't think it's that much harder to get one to two levels higher in abstraction. It doesn't know level 7 is any different from level 6. It just needs the data and compute to start modeling at that level.

In the past you could've convinced me a different architecture is needed to accomplish that. But I also wouldve doubted it could get this far - why would something stop it now?

It just needs to study more long form work and how to reach a conclusion in an essay starting from paragraph 1.