undefined | Better HN

0 pointsbjt123459mo ago0 comments

The problem seems to be two-fold.

Firstly, the capex is currently too high for all but the few.

This is a rather obvious statement, sure. But the impact is a lot of companies "have tried language models and they didn't work", and the capex is laughable.

Secondly, there's a corporate paralysis over AI.

I received a panicky policy statement written in legalaise forbidding employees from using LLMs in any form. Written both out of a panic regarding intellectual property leaking but also a panic about how to manage and control staff moving forward.

I think a lot of corporates still clutch at this view that AI will push the workforce costs down and are secretly wasting a lot money failing at this.

The waste is extraordinary, but it's other peoples money (it's actually the shareholders money) and it's seen as being all for a good cause and not something to discuss after it's gone. I can never get it discussed.

Meanwhile, at a grass roots level, I see AI is being embraced and is improving productivity, every second IT worker is using it, it's just that because of this corporate panicking and mismanagement, it's value is not yet measured.

0 comments

bawolff9mo ago

> Firstly, the capex is currently too high for all but the few.

> This is a rather obvious statement,

Nobody is saying companies have to make LLMs themselves.

SASS is a thing.

bjt12345OP9mo ago

By SAAS I assume you mean public LLMs, the problem is the hand-wringing occurring over intellectual property leaking from the company. Companies are actually writing policies banning their use.

In regards to Private LLMs, the situation has become disappointing in the 6 months.

I can only think of Mistral as being a genuine vendor.

But given the limitations in context window size, fine tuning is still necessary, and even that requires capex that I rarely see.

But my comment comes from the fact that I heard from several sources, smart people say "we tried language models at work and it failed".

However in my discussion with them, they have no concept of the size of the datacentres used by the webscalers.

singron9mo ago

It's not clear to me that fine-tuning is even capex. If you fine tune new models regularly, that's opex. If you mean literally just the GPUs, you would presumably just rent them right? (Either from cloud providers for small runs or the likes of sfcompute for large runs) Or do you imagine 24/7 training?

tdeck9mo ago

This is a good reminder that every org is different. However some companies like Microsoft are aggressively pushing AI tools internally, to a degree that is almost cringe.

throwaway20379mo ago

I don't want to shill for LLMs-for-devs, but I think this is excellent corporate strategy by Microsoft. They are dog-fooding LLMs-for-devs. In a sense, this is R&D using real world tests. It is a product manager's dream.

The Google web-based office productivity suite is similar. I heard a rumor that at some point Google senior mgmt said that nearly all employees (excluding accounting) must use Google Docs. I am sure that they fixed a huge number of bugs plus added missing/blocking feature, which made the product much more competitive vs MSFT Office. Fifteen years ago, Google Docs was a curiosity -- an experiment for just how complex web apps could become. Today, Google Docs is the premiere choice for new small businesses. It is cheaper than MSFT Office, and "good enough".

singron9mo ago

Google docs has gotten a little better in that time, but it's honestly surprisingly unchanged. I think what really changed is that we all stopped wanting to layout docs for printing and became happier with the simpler feature set (along with collaboration and distribution).

bjt12345OP9mo ago

But this is often a mixture of these two things.

The tools are often cringe because the capex was laughable. E.g. one solution, the trial was done using public LLMs and then they switched over to an internally built LLM which is terrible.

Or, secondly, the process is often cringe because the corporate aims are laughable.

I've had an argument with a manager making a multi-million dollar investment in a zero coding solution that we ended up throwing in the bin years later.

They argued that they are going with this bad product because "they don't want to have to manage a team of developers".

They responded "this product costs millions of dollars, how dare you?"

How dare me indeed...

They promptly left the company but it took 5 years before it was finally canned, and plenty of people wasted 5 years of their career on a dead-end product.

j / k navigate · click thread line to collapse

0 pointsbjt123459mo ago0 comments

The problem seems to be two-fold.

Firstly, the capex is currently too high for all but the few.

This is a rather obvious statement, sure. But the impact is a lot of companies "have tried language models and they didn't work", and the capex is laughable.

Secondly, there's a corporate paralysis over AI.

I think a lot of corporates still clutch at this view that AI will push the workforce costs down and are secretly wasting a lot money failing at this.

0 comments

bawolff9mo ago

> Firstly, the capex is currently too high for all but the few.

> This is a rather obvious statement,

Nobody is saying companies have to make LLMs themselves.

SASS is a thing.

bjt12345OP9mo ago

By SAAS I assume you mean public LLMs, the problem is the hand-wringing occurring over intellectual property leaking from the company. Companies are actually writing policies banning their use.

In regards to Private LLMs, the situation has become disappointing in the 6 months.

I can only think of Mistral as being a genuine vendor.

But given the limitations in context window size, fine tuning is still necessary, and even that requires capex that I rarely see.

But my comment comes from the fact that I heard from several sources, smart people say "we tried language models at work and it failed".

However in my discussion with them, they have no concept of the size of the datacentres used by the webscalers.

singron9mo ago

tdeck9mo ago

This is a good reminder that every org is different. However some companies like Microsoft are aggressively pushing AI tools internally, to a degree that is almost cringe.

throwaway20379mo ago

singron9mo ago

bjt12345OP9mo ago

But this is often a mixture of these two things.

The tools are often cringe because the capex was laughable. E.g. one solution, the trial was done using public LLMs and then they switched over to an internally built LLM which is terrible.

Or, secondly, the process is often cringe because the corporate aims are laughable.

I've had an argument with a manager making a multi-million dollar investment in a zero coding solution that we ended up throwing in the bin years later.

They argued that they are going with this bad product because "they don't want to have to manage a team of developers".

They responded "this product costs millions of dollars, how dare you?"

How dare me indeed...

They promptly left the company but it took 5 years before it was finally canned, and plenty of people wasted 5 years of their career on a dead-end product.

j / k navigate · click thread line to collapse