undefined | Better HN

0 pointstakeda3y ago0 comments

Isn't GPT essentially tool for rephrasing what it finds on the Internet, it doesn't really think?

0 comments

It can do some thinking. You can give it instructions to modify a piece of code that definitely isn't on the internet with several steps and it attempts to follow instructions, which, for a human, requires formulating what steps to take.

The prompts have to read like good written requirements for something, so they have some degree of specificity.

But the fact that it can follow instructions and carry them out almost certainly could be considered some form of thinking, especially on novel text not on the internet.

ChatGTP3y ago

Uh, it’s not following instructions, you’re altering you’re prompts until it finds the combination you expect to see…

jazzyjackson3y ago

It is a internet-commenter-simulator, exactly what the world needs right now /s

creatonez3y ago

No. It is modelling the various text generation processes that lead to the contents of the internet. Some of that modelling could absolutely involve "thinking", for processes that involve human thinking.

antoineMoPa3y ago

Have any source to demonstrate that claim?

creatonez3y ago

It's self-evident that GPT is a world-modeller, at least within the confines of the text boundary. It's able to come up with novel ideas seen nowhere in the training data, combinations that demonstrate there is a world concept web and not just a text probability web. It may not "understand" much of the hallucination nonsense it spits out, but there absolutely are moments where it "understands".

See the Rome example on this page: https://oneusefulthing.substack.com/p/feats-to-astonish-and-... This is essentially a completely novel answer to an /r/AskHistorians style question, which I would consider one of the most difficult types of internet text to model, in terms of the amount of understanding and concept webs you need to tie together

Here's another example of GPT-4 doing non-trivial world modelling: How would three philosophers review the TV show Severence? https://i.imgur.com/FBi31Qw.png

PeterisP3y ago

The Othello-GPT experiment (https://thegradient.pub/othello/) probably still is the most relevant argument about these models' capabilities of building an internal world model.

Jeff_Brown3y ago

Doesn't that "It is modelling the various text generation processes that lead to the contents of the internet," seems tautologically true?

(I'm not the person who wrote the grandparent of the present comment.)

j / k navigate · click thread line to collapse

0 comments

vsareto3y ago

The prompts have to read like good written requirements for something, so they have some degree of specificity.

But the fact that it can follow instructions and carry them out almost certainly could be considered some form of thinking, especially on novel text not on the internet.

ChatGTP3y ago

Uh, it’s not following instructions, you’re altering you’re prompts until it finds the combination you expect to see…

jazzyjackson3y ago

It is a internet-commenter-simulator, exactly what the world needs right now /s

creatonez3y ago

antoineMoPa3y ago

Have any source to demonstrate that claim?

creatonez3y ago

Here's another example of GPT-4 doing non-trivial world modelling: How would three philosophers review the TV show Severence? https://i.imgur.com/FBi31Qw.png

PeterisP3y ago

The Othello-GPT experiment (https://thegradient.pub/othello/) probably still is the most relevant argument about these models' capabilities of building an internal world model.

Jeff_Brown3y ago

Doesn't that "It is modelling the various text generation processes that lead to the contents of the internet," seems tautologically true?

(I'm not the person who wrote the grandparent of the present comment.)

j / k navigate · click thread line to collapse