undefined | Better HN

0 pointsoptimalsolver1y ago0 comments

>Frontier Math (25% on high compute, previous 2%)

This is so insane that I can't help but be skeptical. I know FM answer key is private, but they have to send the questions to OpenAI in order to score the models. And a significant jump on this benchmark sure would increase a company's valuation...

Happy to be wrong on this.

0 comments

upghost1y ago

Nope, makes sense to me. Seems unreasonable to conclude the dataset is not compromised now.

cowl1y ago

the question is whether that 25% jump is also because of the compromised first test.

bwfan1231y ago

viewed from a skeptical lens of incentives:

openai and epochai are both startups with every incentive to peddle this narrative. when no one else can independently verify.

j / k navigate · click thread line to collapse