For companies that are built off these AI services, there's a very real fear that the prices we're getting on API calls aren't covering the costs of the requests themselves under some assumption that costs will magically go down, or we'll get locked in.
I hope the companies like Mistral and OpenAI know that if they're selling API access below cost they could be the origin of a __lot__ of companies closing and choose to make sure their operating costs are sustainable. (I understand creating new features and new models taxes additional capitol and that's a perfectly fine use of VC money. But selling a $1.00 for $0.90 is dangerous and bad.)
Don't actually spend the business effort to use them if it's not worth it, but have a business continuity plan to be able to switch over, especially if a "single source provider" going out of business or changing it's pricing could represent a existential thread to your business.
It's the same as any other tech vendor.
So at least for current gen, you're ok.
So, in most cases they have no moat. They are going to close, and while some will close due to operational cost issues as you identified, a lot more will close since they have no defensible business/revenue model.
You can actually see it in paid, web stacks. There’s free ones, esp FOSS, that are really good. So, the paid contenders try to be better at other things. There’s many of them in the market, too, despite over a decade of free alternatives.
If it's the latter, way more startups on SOTA models will become unviable once the investor subsidies dry up.
I'm guessing that's going to be the needle that finally pops this bubble, because as neat as a lot of shit going on with various models is, nobody is going to pay what it actually costs to run the things in order to have that neatness.
You're going to have to bring some actual numbers on that speculation.
(for example, deepinfra has wizardLM-2-8x22B at $0.65/1M output tokens, compared to $6/1M output tokens for 8x22B by Mistral - and of course Mistral has some more expensive, closed source models that perform better)
Imo. A company can use whatever wrapper services but the business they serve should be smarter.
It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Uber has been very successful at it. Amazon has had great success. It's standard.
Investors know what they're doing and know that some companies won't be able to sustain it and will go out of business. It's a calculated, known risk.
And users don't really need to be concerned. Just enjoy the low prices while they last. If the service you use goes out of business, it's pretty easy to switch to another.
> It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Because once they have the market share, they raise prices and/or make the product worse in order to get their money back.
> And users don't really need to be concerned. Just enjoy the low prices. If the service you use goes out of business, it's pretty easy to switch to another.
Well that's the question, isn't it? If the market stays healthy and they don't take the oxygen out of the room and kill the competition and switching costs stay low, then yes by all means enjoy the free ride but plan for its end. But... do plan for it.
If I launch a business that makes sense given the current pricing of LLMs, and the price then goes up by a factor of ten, my own unit economics may not make sense any more.
There's an Uber comparison here: back when Uber was cheap, some people made life decisions about where they lived and worked that were predicated on being able to afford a commute in an Uber (since public transport doesn't serve a lot of places, at least in the US).
This hurt a lot when the prices went up a few years later!
Right but it means you can't depend on that price remaining low. If it's below cost, the price must rise. That may be important for anyone building businesses on top.
> it's pretty easy to switch to another.
Well that assumes the others have true lower prices.
As a consumer it should definitely concern you if you're using poorly financed or unsustainable products. Maybe it's great now but I would rather have a slightly more expensive product that is more stable than one that suddenly had to make the experience a lot worse because they burned a lot of cash and had a down round.
I'm not sure how well that works if vendors differentiate themselves through proprietary models and backend integrations. As far as I know, Mistral only releases weights for some of its models, with others being proprietary.
https://en.wikipedia.org/wiki/Enshittification
This is how the world gets shittier. It's a whole thing. It's a large part of why things are as bad as they are.
But hey, yeah, don't worry about it, they obviously know what they're doing.
Economically rational for the business owner, maybe, but it inevitably results in eventual enshittification for the consumer. (And Uber isn't a great example of this; its share price has only recently surpassed what it was at IPO 4 years ago, and many companies don't even do that well)
Mistral AI employee: https://x.com/dchaplot/status/1800601722722935153
TechCrunch: https://techcrunch.com/2024/06/11/paris-based-ai-startup-mis...
Wall Street Journal: https://www.wsj.com/tech/ai/french-startup-mistral-ai-raises...
I'm working on a local product (coming soon™) that uses Mistral-7B-Instruct-v0.2 under the hood. Parsing screen reader text (and taking actions, etc.) via local LLMs is, in my humble opinion, the future of computing. Even though getting it right is hard, and there's lots of edge cases, it's pretty awesome when it works. Mistral has been (by far) the most reliable model to date. I'm likely going to be fine tuning it over the next few months, but here it is working on Windows and MacOS in both web browser[1] and file manager[2] contexts.
[1] https://vimeo.com/931907811
[2] https://dvt.name/wp-content/uploads/2024/04/image-11.png
For reference: https://en.wikipedia.org/wiki/Mistral_(wind)
Bizzare
