I think we shouldn't let perfect be the enemy of good, here. Yes, ability measurement is sort of fuzzy and it's not perfect. It at least gives you a starting point.
There would be no doubt in anybody's mind that we shouldn't take a kid whose mathematical abilities are counting to 10, and adding and subtracting one digit numbers, and toss them into a calculus course.
So given that there is some boundary, that ability is meaningful in _some_ way, we can start to narrow it down. We can think about methods for measuring ability. What knowledge and understanding is required before you can tackle algebra II?
Again, not perfect, but certainly necessary and better than nothing.