You're exactly right, the exact level of qualification is impossible to quantify. Therefore, we can really only go with rough approximations; this person has experience in the field before, or has relevant education, or has done a lot of similar work before in hobby projects, or whatever.
From what I understand, the suggestion is that if one person is obviously more qualified than another, you pick the most qualified person regardless of other factors, but if there are multiple people where it's unclear who is better, you pick the person who's from a marginalized group.
If we had some kind of ultra-precise scientific way to measure a person's level of qualification, the point would be moot; it would be highly unlikely that two people would measure exactly the same, so we would never be in a situation where we choose based on other factors than qualifications. It's precisely because it's impossible to determine exactly how qualified someone is (probably by the very nature of what we mean by "qualified") that the idea has any merit at all.
