undefined | Better HN

0 pointsfkyoureadthedoc5mo ago0 comments

> What's the benefit to running LLMs locally?

At work:

That I don't rent $30,000 a month of PTUs from Microsoft. That I can put more restricted data classifications into it.

> LLM inferencing isn't particularly constrained by Internet latency

But user experience is

0 comments

kcb5mo ago

30k in a month is an enormous amount of tokens with Claude through AWS Bedrock. And companies already commonly trust AWS with their most sensitive data.

j / k navigate · click thread line to collapse