1. Many humans don't have an idea of the limits of their competence. It's called the Dunning–Kruger effect.
2. LLMs regularly tell me if what I'm asking for is possible or not. I'm not saying they're always correct, but they seem to have at least some sense of what's in the realm of possibility.
1. Dunning-kruger effect describes difference in expected and real performance. It is not saying that humans confidently give wrong answers if they do not know correct ones.
2. That is not my experience. Almost half of the time LLM gives wrong answer without any warning. It is up to me to check correctness. Even if I follow up it often continues to give wrong answers.
1. "It is not saying that humans confidently give wrong answers if they do not know correct ones." And I didn't say that they do either, so you might have hallucinated that.
2. What are you arguing about? I didn't say they're always correct obviously.
