OK,

> remember that Dunning-Kruger isn't that lower skilled people are bad at estimating their skill, it's that they systematically overestimate their skill

The way I see it these aren't very different - conditioning on low skill and randomly sampling will tend to give way more overestimates than underestimates. What is the distinction between being bad at the skill and bad at estimation, and being bad at the skill and systematic overestimation?

> The fact that the artifact is seen in such data is a powerful demonstration that it is not evidence of the Dunning-Kruger effect

"Such data" refers to a world where everyone have absolutely no idea how good or bad they are. To me that is a much stronger argument than that made by DK. So perhaps our differences all come down to our priors. My prior belief (before looking at any data) is that people would know how good they are at a certain skill. If your prior is to expect that people don't know how good they are, then your arguments make sense to me. If however your prior is that people do know how good they are, but are also biased (all in the same direction), then I don't understand how the random data experiment reveals anything relevant to your beliefs.

Some formulation of it can still be true - albeit in a rather uninteresting way.