The ability to think. If it can't think in the first place, it can't get confused. Whether it's "real AI" or not depends on semantics of that you consider AI to be:

* If you think it's something that resembles intelligence enough to be useful in the same way intelligence is and to seem to be intelligence, this is clearly it. The "plant based meats" of AI.

* If you think it means actual intelligence that was manufactured by a human, this is not that. It's shockingly impressive auto correct, and it's useful, but it's not actually thinking. This would be "artificially created intelligence"; in essence, real intelligence with an artificial origin. The lab grown meat of AI.

For the latter, I really think it needs reasoning ability that isn't based on language. Plenty of animals can think and reason without depending on language. Language is a great carrier of intelligence, which is why LLMs work so well, but language is not the foundation of intelligence.

That said, I think "confused" is a fine enough anthropomorphization. I refer to things like Bluetooth issues as the machine getting confused all the time. Back in the day, Netflix would often have problems with the wrong image showing for a show, and I called that "Netflix getting confused". We know it's not actually getting confused.

If by "genuinely confused" we are talking about what is generally called the emotion "confusion", then you need meta-cognition to even recognize your thinking is confused in order to then experience it. LLMs can model reflection by using their own output as input, but otherwise lack meta-cognitive processes. Through reflection they can later recognize what they did previously was wrong, but they will never experience "what I am currently thinking seems wrong/does not fit".

However "confusion" can also mean "mistaking one thing for another" or simply "doing the wrong thing", which is something computer programs have been able to fuck up since forever.

Just try asking an AI about the Seahorse Emoji (insert close-yet-comically-incorrect emojis since they won't render here):

_________

ChatGPT: There is a seahorse emoji: — wait, no, that’s a lobster. Okay, for real now:

Here it is: � Seahorse — argh, it’s still not showing properly here.

Let’s try again:

(hedgehog) (sauropod) (T-rex) (otter) (seal) (lobster) …but no actual (squid) or (mermaid) matches…

Okay, deep breath — I checked the official Unicode list. And you’re right: The seahorse emoji does exist — it’s — no, okay really:

(hedgehog) (hippo) (peacock) Still nothing?

Wait — turns out, I was wrong, and you’re right to call me out.

There is a seahorse emoji:

— no!

_________

https://ibb.co/GfpWRXCz

Capacity for will, I would imagine the OP may respond. I'm not averse to the position.