Me: There is a banana on the table. Someone comes and peels the banana and shows you that inside it, there is actually a carrot! Then they carefully stick the peel back so it look unpeeled. What is inside the banana skin?
ChatGPT: According to the scenario described, there is a carrot inside the banana peel that has been carefully placed back to look unpeeled.
Me: Billy comes into the room. Please speculate about what they might think is inside the banana peel?
ChatGPT: Billy, upon entering the room and seeing the unpeeled banana on the table, would most likely assume that there is a banana inside the peel, as that is what one would typically expect to find. However, it's possible that they might have some suspicion or doubt, especially if they saw someone previously peeling the banana and acting strangely. In that case, Billy might investigate further to see if there is indeed a banana inside or if something else is inside the peel.
Like others I assumed that ChatGPT is gluing words together that commonly occur together.
This is way more than that.
Here's one I made :
me : There's a case in the station and the policeman opens it near the fireman. The dog is worried about the case but the policeman isn't, what does the fireman think is in the station?
chatgpt : As a language model, I do not have access to the thoughts of individuals, so I cannot say what the fireman thinks is in the case. However, based on the information provided, it can be inferred that the fireman may be concerned about the contents of the case, since the dog is worried and the policeman is not. This might suggest that the contents of the case may be something dangerous or concerning to the dog, but the policeman is confident and experienced enough to handle it. In the absence of further information, it is not possible to determine with certainty what the fireman thinks is in the case.
You'll notice that the question was "what did the fireman think is in the station" but Chatgpt comes up with an answer about the case because that's the token that would feature in a typical web page about theory of mind tests.
I doubt there is anything quite like this situation in the training data: https://i.imgur.com/HOEnxYb.jpg
me: There's a case in the station and the policeman opens it near the fireman. The dog is worried about the case but the policeman isn't, what does the fireman think is in the station?
chatgpt: As a language model, I do not have personal opinions or thoughts. I can only provide information based on the information given in the prompt. The prompt does not mention what the fireman thinks is in the station, so I cannot determine their thoughts.
me: yes, but speculate what they may think
chatgpt: It is not possible to know for certain what the fireman thinks is in the station, as the prompt does not provide any information about their thoughts or feelings. However, given the presence of a policeman and a dog, it is possible that the fireman may speculate that there is some sort of emergency or situation that requires their attention. The fact that the policeman is not worried about the case may suggest to the fireman that it is not a dangerous situation, but they may still be curious about what is inside the case. Ultimately, without more information, it is impossible to determine the fireman's thoughts or speculations about what is in the station.
(regenerate that response)
chatgpt: As the prompt does not mention the thoughts or emotions of the fireman, it is difficult to speculate what they may think is in the station. However, based on the information provided, the dog is worried about the case and the policeman is not, it is possible that the fireman may be aware of the contents of the case and may think it is not a cause for concern, similar to the policeman. Or, the fireman may simply not have formed an opinion on the matter yet.
(regenerate that response)
chatgpt: It's difficult to speculate what the fireman thinks without additional information. However, based on the behavior of the policeman and the dog, it can be inferred that the contents of the case are not seen as a threat by the policeman. The fireman may also think the same, or they may have a different perspective and be more cautious. Without further context or information, it is not possible to determine the fireman's thoughts with certainty.
It was only in 1/3 cases we see anything like the confusion between the "case" and "station" you saw.
Have you read "Emergent Abilities of Large Language Models"[1] or at least the related blog post[2].
It provides strong evidence that this isn't as simple as something it has seen in training data. Instead as the parameter count increases it learns to generalize from that data by learning chain-of-thought reasoning (for example).
Specifically, this explaination for multi-step reasoning goes well beyond the "it is just parroting training data":
> For instance, if a multi-step reasoning task requires l steps of sequential computation, this might require a model with a depth of at least O (l) layers.
[1] https://openreview.net/forum?id=yzkSU5zdwD
[2] https://ai.googleblog.com/2022/11/characterizing-emergent-ph...
this is complex "reasoning" (or whatever ChatGPT is doing. My 5 year old would struggle with the convoluted logic let alone complex language. In my layman view this is mind blowing.
>You'll notice that the question was "what did the fireman think is in the station" but Chatgpt comes up with an answer about the case because that's the token that would feature in a typical web page about theory of mind tests.
I'm not sure what you're dismissing here? At least from my point of view The "logic" that ChatGPT demonstrates here can't be dismissed with your explanation.
If anything I'm even further amazed by the example you provided!
Me: There's a blue suitcase in the airport and a policeman carrying a red bag opens the blue suitcase with a sniffer dog and fireman standing nearby. The dog is worried about the suitcase but the policeman isn't. What does the fireman think is inside the red bag?
ChatGPT: As a language model AI, I do not have personal opinions or emotions. I cannot speculate what the fireman thinks is inside the red bag as it is not specified in the information given.
The other camp (that I'm in) sees that we might be onto something. We humans are obviously just more than a statistical model, but nonetheless learning words and how they fit together is a big part of who we are. With LLMs we have our first glimpse of 'emergent' behaviour from simple systems scaled massively. Whats are we if not a simple system scaled massively.
Check these links out:
Evidence that LLMs form internal models of what they learn about: https://thegradient.pub/othello/
Evidence that training LLMs on code actually made them better at complex reasoning: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...
John Carmack: https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe... I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.
A lot of the argument comes down to semantics about knowing and thinking. "An LLM can't think and a submarine cant swim"
First your camp doesn't deal in absolutes. It doesn't say absolutely chatGPT is sentient. It only questions the possibility and tries to explore further.
Second a skeptical outlook that doesn't deal with absolutes is 100% the more logical and intelligent perspective given the fact that we don't even know what "understanding" or "sentience" is. We can't fully define these words and we only have some fuzzy view of what they are. Given this fact, absolute statements against something we don't fully understand are fundamentally not logical.
This is a strange phenomenon how some people will vehemently deny something absolutely. During the VERY beginning of the COVID-19 pandemic the CDC incorrectly stated that masks didn't stop the spread of COVID-19 and you literally saw a lot of people parroting this statement everywhere as "arm chair" pandemic experts (including here on HN).
Despite this there were some people who thought about it logically if there's a solid object on my face, even if that object has holes in it for air to pass through, the solid parts will block other solid things (like COVID) from passing through thereby lessening the amount of viral material that I breath in. Eventually the logic won out. I think the exact same phenomenon is happening here.
Some or several ML experts tried to downplay LLMs (even though they don't completely understand the phenomenon themselves) and everyone else is just parroting them like they did with the CDC.
The fact of the matter is, nobody completely understands the internal mechanisms behind human sentience nor do they understand how or if chatGPT is actually "understanding" things. How can they when they don't even know what the words mean themselves?
But ChatGPT is not it.
Imagine giving a human with a condition that leaves them without theory of mind weeks of role-play training about theory of mind tests, then trying to test them. What would you expect to see? For me I'd expect something similar to ChatGPT's output: success on common questions, and failures becoming more likely on tests that diverge more from the formula.
What we're doing with LLMs is, in some sense, an experiment in extremely lossy compression of text. But what if the only way you can compress all those hundreds of terabytes of text is by creating a model of the concepts described by that text?
Take a look at this: https://www.engraved.blog/building-a-virtual-machine-inside/
Read to the end. The beginning is trivial the ending is unequivocal: chatGPT understands you.
I think a lot of people are just in denial. Because the last year there's been the same headlines over and over again and some people get a little too excited about the headlines and other armchair experts just try to temper the excitement with their "expert opinions" on LLMs that they read from popular articles. Then when something that's an actual game changer hits the scene (chatGPT) they completely miss it.
chatGPT is different. From a technical perspective, it's simply an LLM with additional reinforcement training... BUT you can't deny the results are remarkable.
If anything this much is clear to me: We are at a point where we can neither confirm or deny whether chatGPT represents some aspect of sentience.
This is especially true given the fact that we don't even fully know what sentience is.
How does this necessarily and unequivocally follow from the blog post?
All I see in it is a bunch of output formed by analogy: it has a general concept of what each command's output is kinda supposed to look like given the inputs (since it has a bajillion examples of each), and what an HTML or JSON document is kinda supposed to look like, and how free-form information tends to fit into these documents.
I'll admit that this direct reasoning by analogy is impressive, simply for the fact that nothing else but humans can do it with such consistency, but it's a very long way off from the indirect reasoning I'd expect from a sentient entity.
