If you presuppose that giving disadvantaged groups an extra chance is a positive (your argument sort of does already) and only use wealth as a factor, isn't it still a net positive to uplift the typically poorer group? Doesn't that rightfully uplift more people than it does "wrongly"?