bullshits an answer with confidence (all llms do this)

stupid monty hall

Suppose you're on a game show, and you're given the choice of three transparent doors...

stupid river crossing

A farmer with a wolf, a goat, and a koala must cross a river by boat....

basically, these LLMs have ingested canned solutions and cant reason with newly defined concepts. Anything "out-of-the-box" and they BS canned answers - like the rote student. The BS is particularly distasteful because of the confidence projected in the answer...

So, they are great for looking-up commonly understood "in-the-box" narratives, but are poor at reasoning where there is some novelty. this is what we can expect from a probabilistic "deep" autocompleting machine. unlike a child which can learn ideas and metaphors from a few examples and anomalies.