The Secret Algorithm That's Keeping Students Out of College (opens in new tab)

(wired.com)

14 points_bxg15y ago15 comments

15 comments

8 comments · 4 top-level

ppod5y ago· 3 in thread

>grades from past grads at their school to predict what they would have scored had the pandemic not prevented in-person tests

Wait, this is like reverse affirmative action, no? Students at schools that previously performed badly will have their grades downweighted?

chongli5y ago

Students at schools that previously performed badly will have their grades downweighted?

The article does not make it easy to understand how the actual model works. According to the IBO [1]:

We will use a calculation that is based on the relationship between coursework marks, predicted grades and subject grades to estimate the subject grades candidates would have received if the exams had gone ahead. If the relationship between these elements shows that in previous sessions candidates globally tended to achieve higher outcomes on their exams than their coursework, the calculation used this session will reflect that.

You can call this "reverse affirmative action", but really it's anti-grade-inflation. If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher.

So why would one school underperform on coursework and another underperform on exams? Grade inflation. Lots of schools make their coursework easier to try and boost the chances their students get into university. The whole point of standardized testing is to counteract that.

So in this case, it may be very unfair to some students who might have been outliers on the exam, but there's no way of knowing that without actually running the exam.

[1] https://ibo.org/news/news-about-the-ib/covid-19-coronavirus-...

s1artibartfast5y ago

> If you attend a school where students historically perform much better on coursework than on the exam, your predicted score on the exam will be lower. On the other hand, if your school tends to perform better on the exam than in coursework, then your predicted score on the exam will be higher.

This is terrifying. It works for the average student, but distorts the performance for any student who isn't typical or better than their school.

It seems that it would be better to just ignore the testing component of the IB score this year when judging students. Instead they modeled what the students would have gotten on the test, and then judged them based on their modeled performance.

1 more reply

xyzal5y ago

I understood it so as well. It seems to be the most horrible application of machine learning I have heard of lately.

quantified5y ago· 1 in thread

The article doesn’t include any commentary related to students that were mis-predicted in the other direction, i.e higher-than-expected scores. If the results stand, they’ll be an interesting case study in the predictive value of IB scores on college performance.

MattGaiser5y ago

I suspect nobody is willing to go on record about that.

prepend5y ago

In the future I just have to do really well in kindergarten and all the rest of my school and career will be predicted by a ML model with no correction whatsoever.

I’m an IB grad and 5/6 exams could have been done electronically. Should at least give students the option to take exams.

naveen995y ago

My 13 year old son just started doing classical mechanics on ocw.mit.edu No one can keep students out of open courseware. signaling is a zero sum game, but there is always a shortage of students who have the interest and aptitude to learn.

j / k navigate · click thread line to collapse