undefined | Better HN

0 pointsimiric4mo ago0 comments

Or, 2b: the nerf is real, but benchmarks are gamed and models are trained to excel at them, yet fall flat in real world situations.

0 comments

I mostly stay out of the LLM space but I thought it was an open secret already that the benchmarks are absolutely gamed.

j / k navigate · click thread line to collapse

0 pointsimiric4mo ago0 comments

Or, 2b: the nerf is real, but benchmarks are gamed and models are trained to excel at them, yet fall flat in real world situations.

I mostly stay out of the LLM space but I thought it was an open secret already that the benchmarks are absolutely gamed.

j / k navigate · click thread line to collapse