Imagine subjecting a random human to the same battery of conversations and judging the truthfulness of their answers.

Now, imagine doing the same to a child too young to have had many years of reinforcement of the social consequences of not clearly distinguishing fantasy from perceived truth.

I do think a human adult would (still) be likely to be overall better at distinguishing truth from fiction when replying, but I'm not at all confident that a human child would.

I think LLMs will need more reinforcement from probing the limits of their knowledge to make it easier to rely on their responses, but I also think one of the reasons people hold LLMs to the standard they do is also that they "sound" knowledgeable. If ChatGPT spoke like a 7 year old, nobody would take issue with it making a lot of stuff up. But since ChatGPT is more eloquent than most adults, it's easy to expect it to behave like a human adult. LLMs have gaps that are confusing to us because the signs we tend to go by to judge someones intelligence are not reliable with LLMs.

Paradoxically, it seems as if the people who are pushing this the hardest are the same people who flat out deny even the slightest flicker of what could be considered intelligence.

If we made a ChatGPT that has some “senses” it can use to verify its perceptions, and it does as well with this as humans generally do, I’m sure we’ll get interesting results but I’m not sure we’ll be any closer to solving the problem.