undefined | Better HN

0 pointsEvgeniyZh2y ago0 comments

Note that current GPT-4 pass@1 for HumanEval is closer to 90% than to 67% reported in GPT-4 technical report, as reported, e.g., in [1]

0 comments

Unfortunately, we have no idea whether GPT-4 has been further trained or finetuned on contaminated data since then.

Good point, I guess Meta should be using that number in their chart

j / k navigate · click thread line to collapse

0 pointsEvgeniyZh2y ago0 comments

Note that current GPT-4 pass@1 for HumanEval is closer to 90% than to 67% reported in GPT-4 technical report, as reported, e.g., in [1]

Unfortunately, we have no idea whether GPT-4 has been further trained or finetuned on contaminated data since then.

Good point, I guess Meta should be using that number in their chart

j / k navigate · click thread line to collapse