That doesn't mean they aren't useful.
The primary use of a number like Adversity Score is to create a pretty spreed sheet that generates pretty graphs that look good in board meetings, grant applications and pamphlets. While the data backing up the graph is impeccable it also is built on layer after layer of imperfect abstractions until the graph has very little to do with reality.
There are neighborhoods full of rich people, middle class professionals, working class people, and slums and we can tell the difference.
I don't know why you think that's so hard. It seems quite easy to me. Certainly worth trying instead of (as you seem to advocate) just giving up at the slightest difficulty.
If we could quantify adversity we could calculate the percentage out of all human suffering that occurred during the Trail of Tears or determine the single most resilient living person.
I'm not saying we should consider adversity in college admissions. I'm saying we shouldn't quantify human emotions and experiences.