If you had an llm that could accurately predict when a claim is uncertain it would be very popular, I think. I would pay for that kind of reliability tbh

Up to the point where consumers notice and decide to stop using these models because of it.

Might be why we're already rarely seeing models output an "I don't know".

According to your logic the market will produce an LLM that consists only of 'PRINT "I don't know."'.