AI can cost more than human workers now (opens in new tab)

The way these agents are being used now is crazy in how inefficient the token utilization is. Reading from ill structured knowledge dumps meant for humans, ralph wiggum loops. Just crazy iterations for simple things.

I think its the same disease that makes people make shitty, unoptimized, bloated apps because modern client machines ahve so much ram. But that wont work AI agents. Not until tokens become dirt cheap anyway. Until then we'll need apps with more efficient usage patterns

I see people highly trained engineers spend hundreds of thousand of tokens doing what can reliably be accomplished with 150 lines of python.

I think the push from management for us to use AI has made it so we don’t have to be efficient with our consumption, so now we write md files which we feed to Claude in a loop instead of python and bash scripts to do routine tasks.

In a sense, everyone is a startup now... At least, every serious user of agents.

So... if you spend $3m to replace a $1m team... you are betting on that $3m cost coming down. It's a proof of concept. The first step is to find out if agents can do the job at all. At this point you are hoping future versions will get more efficient.

Trying to make something efficient before you know that it is even possible is hard.

Drop-in, profitable on day-1 isn't what the frontier looks like.

I had that conversation with our AI VP recently. At a certain cost entertaining humans will be more cost effective from a financial and energy perspective. Especially on a global scale.

“When AI labs raise prices, big spending on AI could shift from a flex to a liability.”

because companies will need

“proof of productivity gains or metrics that show a clear return for all this AI investment.”

which in my opinion is simply not true. I haven’t seen any good study that showed AI to actually improve productivity overall. It massively helps in some areas, but then promptly gets stuck in others. So you still need an expert to guide it.

Thats the thing right, like how many tokens can you extract from an employee per day? In API cost terms, LLMs cant currently cut it.

It gets worse when you look at LLM (or even any other kind of AI) benchmarks, they tend to cap out around 110% of human performance.

The more that LLM services try and creep towards profitability, the more features they are paywalling behind higher tiers, the more some lazy junior dev is going to look like a better value proposition.

And when some of the CTO's they have pushed LLMs on to go looking for cost savings, some of them are going to look at opex instead of capex and in house the LLMs using open models.

The only real question to my mind is whether the air will be let out of the AI balloon slowly, or if it escapes in one big pop.