The thing is, no one has the slightest idea how to stop hallucinations.

The models are fundamentally "hallucinatory" at core - they generate what is _probable to follow the string thus far in its training corpus_, modulo RLHF and friends.

Notice that nothing there has any rigorous relationship to truth.

Sure, the companies could start pumping money into pure research on what models other than transformers might yield something that can reason rigorously, but at that point you're talking about finding a way to throw out LLMs entirely in favor of a less-pathologically-broken model, like Gary Marcus keeps complaining people should be doing.

Rather, they are stuck unable to do that much better, unwilling to admit (especially in a way that might spook shareholders) that it's a hallucination-machine all the way down. They're playing for time and market-share while hoping some unspecified and inherently-unpredictable new discovery arrives which will be compatible with their existing infrastructure and investments.