They get simple, rephrased, but conceptually equivalent questions really wrong and they do this:
1. while the context already contains their previous answer to the original question (which was correct),
2. while the context contains all background information on the topic that would allow an intelligent being to arrive at the correct answer through simple logical deduction,
3. without recognizing or acknowledging that they provided a conflicting answer (without being prompted),
4. while denying that the two answers are contradictory if that fact is pointed out to them,
5. while fabricating a list of bogus reason justifying a different answer if pressed for an explanation.
That's one common failure mode, the other common failure mode is where they uncritically accept our own erroneous corrections even when the correction contain obviously flawed reasoning.
This behavior demonstrates a fundamental lack of conceptual understanding of the world and points at rote memorization in the general case. Maybe LLMs develop a more conceptual understanding of a certain topic when they've been benchmaxxed on that topic? I don't know, I'm not necessarily arguing against that, not today anyway.
But these errors are a daily occurrence in the general case when it comes to any topic they haven't been benchmaxxed for - they certainly don't have a conceptual understanding of cooking, baking, plumbing, heating, electrical circuits, etc.
