Perhaps vibe coding the A/B testing engine isn't the best idea.

Solution: don't A/B test your users.

A/B testing people without their informed consent is immoral, unethical, and should be illegal.

pretty much none of these big providers are offering the guarantees needed to be taken seriously in workplaces right now. the technology itself isn't offering the deterministic guarantees that should warrant it in the workplace right now. problem is everyone's foot is just on the gas. even if your workplace isnt paying for it, people are just straight up rolling their own personal claude accounts to do work at orgs.

ive been trying to make the case all year that if we're going to let employees do shit with ai, lets try claude. in the past like.. 2-3 weeks all that goodwill has basically evaporated.

local inference needs to take off asap because all of these entities actually suck and i wouldn't trust a single sla with anthropic. they are not acting like a serious company right now, this is a joke.

I just cancelled before seeing this news. i was already pissed about constantly hitting limits on the 20 a month plan and looking for alternatives and this seals the deal. Bye bye!

It depends on the kind of work you do.

I'm in the academia, and Claude's performance in my field could be described as a very fast junior grad student. When I use Claude Code, I typically spend a few hours figuring out what needs to be done exactly, and describing it in sufficient detail. Then Claude does it in 30 minutes, while an actual student would need days. And then I spend anything from minutes to days evaluating the results, depending on if it needs to be tested with real data and how much weirdness those tests uncover.

But I also have other work to do beyond guiding the automated grad student. Which means my Claude Code usage rarely exceeds 1–2 hours/week.

I use Pro professionally and didn't hit limits most of the time. I believe I used up 5hr quota once or twice. We switched to Team sub and I'm on Standard(which is Pro x1.25 I believe). I don't vibecode entire applications, I ask it to make boilerplate, smaller, well scoped features or fix some errors. I don't let it go off with a prompt "make another netflix clone" cause I just don't see any real value in that

Just the Pro Plan Claude Code on its own? Maybe you could last a full day on just using Sonnet. Maybe one Opus dab in the morning to plan your Haiku/Sonnet day?

I have Pro Claude, Plus GPT and Pro Gemini. When one runs out I switch to another project on the next LLM. If I really need a task finished I'll restart it on another LLM, but I'm loathe to do that as it eats tokens just getting back up to speed.

I think it's more about how they approach their users in general that is the problem here.

But if Sonnet is bad it would give bad impression of the product, no? And it also takes compute, so you give a bad hallucinating impression of your product while still losing compute.

I mean, this is why they do A/B testing. This way of testing stuff is not new at all, people who act genuinely surprised need to do a reality check. Companies want to maximize profit. They do this by testing what creates the biggest profit. A/B Testing is one of the ways to do this, and it has been used for decades in precisely this way.