cerebras: 450 tokens/sec llama 3.1 70B (opens in new tab)

(theregister.com)

7 pointsdavidfiala1y ago2 comments

2 comments

Cerebras fails the "how many r's in strawberry" test. Grok is the only one who passed that test.

Going to be interesting to see the speed and accuracy keep increasing, cant imagine how fast/accurate things will be in a decade. Cant wait.

- 1,800tps on llama 3.1 8B

- 450tps on llama 3.1 70B

free chat interface is at: https://inference.cerebras.ai (requires login)

j / k navigate · click thread line to collapse

Cerebras fails the "how many r's in strawberry" test. Grok is the only one who passed that test.

Going to be interesting to see the speed and accuracy keep increasing, cant imagine how fast/accurate things will be in a decade. Cant wait.

- 1,800tps on llama 3.1 8B

- 450tps on llama 3.1 70B

free chat interface is at: https://inference.cerebras.ai (requires login)

j / k navigate · click thread line to collapse