No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores. Which is not what we would expect: we would expect that their predictions of their test scores would be strongly (or at least more strongly) correlated with their actual test scores. The question the D-K effect raises is why that is not the case, and it's a valid question--one which this article does not even attempt to answer.
I couldn't access the original 1999 full article because it's behind a paywall, but is this accurate?
The abstract states that people "grossly overestimated their test performance and ability". If it were truly uncorrelated, wouldn't we expect there to be roughly equal number of people who "grossly underestimate" their performance? In any event, I wouldn't expect a weak correlation to be described as "grossly [anything]".
Or maybe this is just the D-K effect in myself trying to understand the abstract :-)
I understand that the words D-K used were along those lines; but the actual data shown in their graphs says what I said. Just look at the lines on the graphs. The "actual" lines are 45 degree lines up and to the right--as they must be. But the "perceived" lines are horizontal, or roughly so. That means the two lines are uncorrelated, or nearly so.
Saying that underperformers grossly overestimated their performance is only describing part of that--the part where the "perceived" lines on the left sides of the graphs are above the "actual" lines. It does not describe the other part--the part where the "perceived" lines on the right sides of the graphs are below the "actual" lines.
D-K don't talk about that at all in their paper, which means their paper itself misdescribes the actual data they found. And if this article had said that, it would have been a valid criticism. But this article is not talking about what D-K said in words; it's talking about D-K's actual graph. And the article's claimed criticism of D-K's graph is not valid: the article is in fact just describing the same graph in different words. It's not offering a different "explanation" of why the graph looks that way; in fact, as I said, it offers no explanation of that at all; it doesn't even recognize that as a question.
Looking at the graphs, isn't the implication that the distance between the two lines is the metric of interest? In other words, that the distance between real/perceived scores in Q1 & Q2 is "grossly" wider than the distance between Q3 & Q4, with the noted exception that only Q4 underestimates?
If estimates of ability were a truly random sample with mean of 75th percentile ability, then the net effect will be vast overestimation. If they're a truly random sample with mean at 50th percentile, it will be equal. At 25th, underestimation.
It's like how 90% of people estimate that they're in the top 50% of drivers by ability.
The article this thread is about links to a PDF of it, I was able to download it just fine:
https://www.avaresearch.com/files/UnskilledAndUnawareOfIt.pd...
but the use of percentile to percentile (or quartiles, but those are just grouped percentiles) to give the impression of a particular kind of effect (lower groups overestimating and higher groups underestimating) is a flaw i think. it's a common one, in my experience, when dealing with percentages.
if you think about it, percentages/percentiles have to be bounded at 0 or 100. for a dunning kruger effect not to appear, participants at both ends of the ability spectrum would have to be eerily accurate in their self-assessments. if they aren't, there's just more space on one side of the measurement scale for each group to make an error (if you score 1 in ability, there's ~99 percentiles available for you to make an overestimate and only 1 to be accurate, ditto for those with high ability. those in the middle of the ability group have equal chance on either side and so appear statistically more accurate even with a purely random distribution of guesses). so if there is any measure of central tendancy towards the middle percentile in the estimations of peoples abilities at all (and i would argue there is a priori reason to believe there would be, as the alternative would require those at both ends of the distribution to be getting increasingly accurate, which would be really weird), then practically any real world graph of percentile performance to percentile estimation will show a dunning kruger effect (with lower ends overestimating and higher ends underestimating). the article does a good job of showing this by plotting just random estimations and observing one appears.
> No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores.
The problem is that in common Internet discussions, there is no fixed meaning of D-K. When things heat up, D-K becomes "his belief in his competent is an indication of his incompetence,ha" but in more calm debate is become "less competent people have a somewhat less accurate understanding of their competence".
In this form, D-K is a classic Motte and Baily device [1]. It's plausible, unsurprising and uninteresting that less competent people in a field self-access somewhat less accurately (the motte). It's completely unsupported but very "juicy" that someone's claim of competence indicates an incompetence (the bailey).
[1] To one more pieces of rhetorical which I'm mention also proves nothing but is illuminating - https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
That's why I focused explicitly on the actual data in the D-K paper and what it actually says.
