This doesn’t seem to apply to AI for some reason. It keeps generating incorrect results after incorrect results, yet people continue to trust its output.
I don’t know what to make of this.
Human trust differs from mathematical trust. And branding / marketing abuses the ambiguity.
There is no shame in a "likely to hallucinate" model that can be instantiated 1,000 times across 1,000 different machines spread throughout our planet. So, human trust is broken by machine trust.
Take a look at the Forbes billionaires list and some of their statements. Or maybe at the politician fact checkers. If only being wrong damaged reputations.
Basically it is a pretty version of a dumbed down partially incorrect answer. With a knowledgeable user it would be very good, but he has no idea he is wrong. I’m not sure what Dunning Kreguer with graphics should be called.
Then, predictably, finding the collection of supporting details + vetting the content in question.
This is an issue we, technology-folk, ought to help guide our non-tech-co-folk through engaging with, BTW. Our responsibility is rising with tech becoming more deeply entrenched / required for society's operations.