Faster inference won't save you (opens in new tab)

(graphcoder.ai)

5 pointsramstar300013d ago4 comments

4 comments

4 comments · 2 top-level

shreyash308713d ago· 1 in thread

The latency table says it all. Cloud-to-cloud is 40ms for 20 turns. Hotel Wi-Fi is 16 seconds. You can halve inference time and still have a broken product on bad connections.

Var137713d ago

is this an LLM?

Var137713d ago· 1 in thread

does this mean you can disconnect from the internet entirely with the agent loop still running?

ramstar3000OP13d ago

yes this is central to our thesis :)

j / k navigate · click thread line to collapse