> There are exceptions: people will produced incorrect post hoc explanations for their behavior under sufficient duress when "I don't know" isn't perceived as an acceptable answer.

Did Khaneman get into this in his book?

Also, as for the point of contention, I think if you tell the LLM to show its work in mathematically formal notation, it's far more likely to be able to produce correct answers (I think there was a post on here in the last week or so demonstrating that?). I think this kind of makes the comparison to humans more fair, because inside their mind humans are doing some sort of intermediate math in their head for anything beyond trivial problems, and an LLM needs to be able to speak explicitly to compete fairly with that (I speculate/propose).