Testing suggests Google's AI Overviews tell millions lies per hour (opens in new tab)

> If the first page of results would contain the answer to your question

You can find complaints on this site, not too long ago, that Google was failing to have good results anymore. I don't feel the ranking has particularly improved since then.

I don't think that "right almost every time" is enough. It's different when you're searching for an answer yourself and you're expecting to dig for right one among others not so right. But when you get one answer from LLM you either trust it or don't. If you do you're bound to be lied to from time to time and face the consequences. If you don't, you're back to searching manually anyway.

I find myself questioning LLMs a healthy habit these days.

Considering the accuracy of journalists in general (contentious subject, but the studies I've seen seem to confirm the gell-mann amnesia effect to some degree), I'd say AI Overview isn't bad at least.