undefined | Better HN

0 pointsfreeqaz3y ago0 comments

I'm on mobile and struggled to compare these two tables properly. Would you mind posting a summary of your findings?

Here are some values but I don't know what they mean. LLama 60B on the left, Cerebras 13B on the right.

PiQA: 82.8 / 76.6 WinoGrade: 77.0 / 64.6 ARC-e: 78.9 / 71.4

0 comments

1 comments · 1 top-level

gpm3y ago

Really short summary: LLaMa is better, even smaller LLaMa models.

Table format: Benchmark, Cerebras 13B, LLama 7B, LLama 13B, LLama 60B

HellaSwag, 51.3, 76.1, 79.2, 84.2

Piqa, 76.6, 79.8, 80.1, 82.8

Wino-Grande, 64.6, 70.1, 73.0, 77.0

Arc-e, 71.4, 72.8, 74.8, 78.9

Arc-c, 36.7, 47.6, 52.7, 56.0

OpenBookQA, 28.6, 57.2, 56.4, 60.2

j / k navigate · click thread line to collapse