undefined | Better HN

0 pointsjoelthelion2y ago0 comments

I wonder how that weird HellaSwag lag is possible. Is there something really special about that benchmark?

0 comments

Tech report seems to hint at the fact that GPT-4 may have had some training/testing data contamination and so GPT-4 performance may be overstated.

1 more reply

erikaww2y ago

yeah a lot of local models fall short on that benchmark as well. I wonder what was different about GPT3.5/4's training/date that would lead to its great hellaswag perf

j / k navigate · click thread line to collapse

0 comments

HereBePandas2y ago

Tech report seems to hint at the fact that GPT-4 may have had some training/testing data contamination and so GPT-4 performance may be overstated.

1 more reply

erikaww2y ago

yeah a lot of local models fall short on that benchmark as well. I wonder what was different about GPT3.5/4's training/date that would lead to its great hellaswag perf

j / k navigate · click thread line to collapse