undefined | Better HN

0 pointsButtons84023d ago0 comments

I think companies will eventually just buy a local AI server.

Using local hardware is expensive when it's running a complicated software stack that can break in 10,000 different ways.

These eventual local AI servers will just talk some protocol for AI and sit in the corner and nobody will think about them.

I guess they still might need access to various systems, so idk. Eventually I think someone will offer "AI in a box" though, running the latest open model or whatever.

0 comments

5 comments · 2 top-level

dangus23d ago· 3 in thread

I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS?

“AI in a box” sounds a heck of a lot like “the box” from the Silicon Valley TV show. Or the Google search appliance. Or name any other on-premise thing that is equally dinosauric.

The real finding of this article is that AI tokens are direct competitors with offshoring. $1,500/month buys you a whole employee in India.

And this is before AI companies inevitably increase pricing after the conclusion of the growth phase.

pm9023d ago

> I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS?

For customer facing, production software, its worth paying a cloud tax to get the reliability guarantee. For tools that are used by engineers for code development, there is no need for such bulletproof guarantees.

dangus23d ago

That makes very little sense. SaaS/cloud tooling is overwhelmingly popular for internal tooling.

Which category of developer tool has on-premise as the more popular option?

Cloud isn’t about “reliability,” it’s about being able to focus on your core business rather than spending all your time maintaining stuff.

Buttons840OP22d ago

Local AI servers are different because they don't have to form a single system. If one AI server goes down, just use the other one.

This is unlike customer facing systems where, if your database server goes down, you probably can't just use the other one--the whole system is down.

1 more reply

pm9023d ago

Yep, its already quite easy to do so with tools like opencode/openrouter. Ive used some open source models and they seem … ok? Im not doing foundational math, just refactoring code, understanding existing code etc. I don’t see a future where companies blow 11% of employee compensation on a single tool; the hosted AI server + oss models will 99% win out.

j / k navigate · click thread line to collapse

0 comments

5 comments · 2 top-level

dangus23d ago· 3 in thread

I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS?

“AI in a box” sounds a heck of a lot like “the box” from the Silicon Valley TV show. Or the Google search appliance. Or name any other on-premise thing that is equally dinosauric.

The real finding of this article is that AI tokens are direct competitors with offshoring. $1,500/month buys you a whole employee in India.

And this is before AI companies inevitably increase pricing after the conclusion of the growth phase.

pm9023d ago

> I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS?

dangus23d ago

That makes very little sense. SaaS/cloud tooling is overwhelmingly popular for internal tooling.

Which category of developer tool has on-premise as the more popular option?

Cloud isn’t about “reliability,” it’s about being able to focus on your core business rather than spending all your time maintaining stuff.

Buttons840OP22d ago

Local AI servers are different because they don't have to form a single system. If one AI server goes down, just use the other one.

This is unlike customer facing systems where, if your database server goes down, you probably can't just use the other one--the whole system is down.

1 more reply

pm9023d ago

j / k navigate · click thread line to collapse