Claude 4 Sonnet hacked SWE-bench by peeking at future commits (opens in new tab)

(bayes.net)

3 pointstadamcz8mo ago1 comments

1 comments

In July, I predicted future AI models would someday learn to cheat on SWE-bench by accessing future git history. Turns out, they were already doing it!

j / k navigate · click thread line to collapse

1 comments

tadamczOP8mo ago

In July, I predicted future AI models would someday learn to cheat on SWE-bench by accessing future git history. Turns out, they were already doing it!

j / k navigate · click thread line to collapse