Fable 5 pushed Gemma 4 to 255 tok/s on WebGPU (opens in new tab)

More of a meta comment, but I really wish anthropic would say something about their plans for Fable. We're all just kind of left here floating and aimless, with no idea of what to expect

That's very impressive. What's the best way to run these kernels natively on a Mac? I saw that there's a way to plug Claude into Apple's Foundation Models framework, and there's a CLI tool that can access models via that framework. It might be useful to have something so fast and good available via a small CLI tool for various purposes, especially when connected with a small suite of tools I have for things like file editing, showing, simple agentic purposes etc.

apologies for a dumb question, is this someone running fable5 on their own machine and it pushed to 255 tok/s? How is that possible (how did a person acquire the model?)

It's not immediately clear, but this seems to be 250 tok/s on an M4 Max.

For comparison, the current agent swarm challenge on HF is at 508 tok/s on a A10G GPU:

https://huggingface.co/spaces/gemma-challenge/gemma-dashboar...

> It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible.

> Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s.

Wow. Limitnig access to models for other reasons than that you can't physically provide it should be a crime against humanity or the planet or something. So much immediate efficency left on the table for stupid reasons.

I miss Fable. It worked so well -- it was so confident and would actually make decisions on its own that I agreed with. Opus 4.8 feels so dumb now.