Memory has grown to nearly two-thirds of AI chip component costs (opens in new tab)

China is about to flood the market and prove this notion wrong. If there is demand they want to meet it with supply.

But to your point, that is exactly how American companies like to play now. No one is stopping them from screwing over the consumer.

I have a Micron near me and they are building another chip facility but we are years away still so I suspect China will beat them to the punch.

This is wrong. It is NOT in their nature to keep the market under-supplied -- eg, Samsung, the industry's largest company, was notorious for expanding their capacity during the industry downturn to gain market share while everyone else was cutting back to minimize loss.

I'm guessing you are also probably unfamiliar with the terms like "chicken game" which refers to the cutthroat, high-stakes price wars where dominant semiconductor manufacturers intentionally overproduce and slash prices. This is literally how the industry went from dozens to just three majors today since the 80's.

If the existing memory makers retains control of the market and don't defect from the optimal-long-term equilibrium for themselves, that's true. It just takes one player to defect for short term gains as we've seen with some past boom-and-bust cycles. Alternatively, it takes a sufficiently-resourced player with enough incentive to enter the market themselves (NVidia, Google, Amazon, the PRC government through one of many companies...)

CXMT is scaling up incredibly fast, they are on a clock (south koreans) their monopoly will end relatively soon, although I'm guessing that the AI companies will crash before that anyways.

Relevant article posted on HN about this a few days ago: https://davidoks.blog/p/ai-is-killing-the-cheap-smartphone

I struggle to think of a line of business as cyclical as DRAM, maybe like certain kinds of mining would be my only thought.

The DRAM fabs have been on a roundabout for 40 years going from getting accused of price fixing and cartel behavior, to struggling to keep the lights on.

And imo it's not really their fault, it's all the lead time of advanced semiconductors, combined with the commodity dynamics of oil. And the goal is to match that supply to the demand of everything from consumer electronics to more datacenters than you can shake a stick at.

It's maddening to try and solve that, so at this point I really don't fault them for prioritizing survival.

Reminds me of how Samsung is giving out $340,000 per person bonuses. Shows you how much of a stronghold they have in market.

What you described only works if the manufacturers agree to price fix. Otherwise, in a free market, they'll race to increase their earnings by meeting the demand.

Supply and demand always balance out. There is no way manufacturers aren’t going to compete away these inflated margins, as long as they feel like this demand is sustainable.

> It is in the nature of their business to keep the market under-supplied

What?! If they did an anti competitive agreement sure. Otherwise no as each supplier is incentivized to produce more than its competitor and less than the demand, while divesting just enough to survive the oversupply risk.

Apple could always decide to build their own fab or some such thing.

These crazy hardware price increases will probably delay everything by at least 2-5 years. Then add at least 5-10 years for all these refinements and optimizations to permeate universally.

Until everything matures, most likely the current iteration of OpenAI and Anthropic will be long gone, along with their current business models.

This has been around for quite some time, to the point I had to read this a couple times to understand what you meant. Mighta predated LLMs even.

That's already the case with say bf16

Notoriously terrible?

Posits do a little better if your numbers are biased enough toward 1, but not much better. A 16 bit posit in a near-ideal situation matches an 18 bit IEEE float, and in a pretty wide range of situations loses to either fp16 or bf16.

Training anything at 8 bits is going to be tough, and it's hard to say if the flexible exponent is worth the precision tradeoffs.

It's very easy to find out when the new fabs come online. Try asking Claude or ChatGPT.

No new DRAM fabs are being built. That's why you don't see any predictions.

Is any of this actually unforeseen? Buying the vast majority of the world’s supply of something does have mostly predictable consequences.

“Unforeseen consequences” in the same way death of the target is when someone aims a loaded gun at their head and pulls the trigger.

Patents is not the issue here. Not even close.

The up-front investment of a memory fab is measured in billions, and takes years to construct and get running. The margin on the chips themselves is terrible, so without scale its not worth even trying. DDR5 is a industry standard that takes some effort to conform to, but the licence fees is a drop in the bucket to the cost of creating a fab.

The fabricators were cautious about increasing production, and slow to start planning. It takes further time to build up capacity, and if the demand drops down, they may end up producing dram at a loss when the market flips over to oversupply. The demand whiplash could kill any company that dared betting on increasing production. See the "bullwhip effect" https://en.wikipedia.org/wiki/Bullwhip_effect which has killed semiconductor fabricators before.

There is a discussion to be had about how to maintain national semiconductor production in Europe and US as a strategic industry, but historic attempts have all failed.

I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?

For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.

Unless there's a new paradigm, scaling up is all they can do to improve performance. They've shrunk down all the way to 1-bit models and all the low-hanging fruit is gone. There's no way for them to get much smaller, so they have to get bigger and faster to meet expectations.

Probably, but at some point we're very likely to run out of significant training improvements and it's not clear that we'll see that point coming from a long way out.

Likewise it's probably dwarfed by improvements in how we make dram - continuing the roughly exponential (maybe a bit less recently) scaling of chips - but not necessarily.

The 2x from returning to previous costs is interesting because it's practically guaranteed, and it's on top of everything else. We're just currently "overpaying" (relative to the stable market price) for the manufacture of dram because of a sudden increase in demand.

For supply to meet demand. Depends very much on how aggressively producers scale and on how demand grows or shrinks.

Safe to say at least a year or two. It'd be shocking if it took a decade.

All the projections I've seen have said that the earliest we might see the curve flatten is 2030.

It just takes that long to get a fab up and running.

CORSAIR Vengeance 96GB (2 x 48GB) SO-DIMM DDR5 5600 CMSX48GX5M1A5600C48

Bought an extra one by accident, paid $218.99 March 2025

Goes for $1400 now. I haven't gotten around to selling it.

Nice, you were lucky. =3

Demand for DDR3 is up because people who want DDR5 or DDR4 but can't afford either any more are choosing DDR3 and old DDR3-compatible systems to put it in, instead of what they really want.

All memory products use many shared resources in the supply chain, so if there is high demand in one product line, others have to raise prices to compete for the resources or stop making those lines altogether.

That is to say at least you were able to buy them at $350 today, with the current trajectory there will be no supply at all in few months.

You could set up swap space on Intel Optane media, it'll be about the same performance as DDR3 and sells for ~$1/GB on the secondary market. Though it will be a lot more power hungry than Flash, let alone DRAM - so not suitable for all uses.

there's an economic term for this: substitute good. https://en.wikipedia.org/wiki/Substitute_good

Being desperate?

Used enterprise HDD’s also jacked up now. It’s absurd lol

People spend that much a month on restaurants

Someone was selling an Epyc machine with 512GB RAM @ 500 EUR last year. I regret not buying it now ...

I forgot to add, I paid ~500 each, Samsung for the same drive is quoting $2k on their site (maybe a new sku). These were bought 2ish years ago. Strange things are a foot at the Circle-K.

The price of SSDs is similarly depressing.

> Its price just goes up and down in cycles.

Historically the price has always trended downward. When I first got into computing $200 could buy you 128 MB (yes M) of ram. Really nice systems had 512 MB.

That's obviously changed over the decades as process shrinks have lead to higher memory density. We should generally expect that ram will cheaper up and until the point where process shrinks stop happening. They've definitely slowed, but they haven't stopped.

> Crucial are already selling DRAM made by CXMT.

Crucial was disestablished this year.

This has already happened. Home PC market is practically dead already due to memory, ssd and graphics card price inflation. Makers of components like PC cases and power supplies etc. are seeing demand down 30-40% year over year and this is going to put many suppliers out of business. NVDIA has stopped even listing gaming revenue on their earnings reports. Both NVDIA and AMD are not seriously interested in supplying the consumer GPU market anymore either.

The only hope left is really Apple, but even apple has conspicuously delayed the launch of M5-gen mac minis and mac studio. Mostly because even Apple can't source enough DRAM to fully supply all their product lines.

Indeed, Gamers Nexus is doing interviews with PC component manufacturers, and some are hurting bad right now. The PC market is no longer in competition, but rather survival mode. =3

https://www.youtube.com/@GamersNexus/videos

there's much more than triple A video-games running at 240 Hz on Ultra settings... a 200 USD laptop/computer has enough power to run hundreds of interesting indie games and AAA from the past

It might kill the console gaming market, too. Typically consoles get cheaper over time post-release. Instead, all the latest gen consoles are getting price hikes and at least one company is potentially pushing back the next gen release (PS6). A PlayStation 5 for $900? I'll just wait and be happy with my perfectly usable Switch 1 (since the 2 is also more expensive than it should be).

I don't understand the threat to the PC market.

Prices haven't risen THAT much and are quite affordable. And if you look at the improved quality of upscalers (DLLS 4.5 for example), gaming is now more affordable than ever, despite the increased cost of components.

Of course, the 5090 prices are insane, as are for SOME memory models, but that's nothing new and represents a fairly small market share.

> When I started building gaming pcs, the top top card was 750$ (NZD)

When I started building gaming PC, the top $700 cards didn't even provide comfortable performance or graphics. Back then, you were supposed to have several of this connected SLI or somethin. And even then, it wasn't always reliable, and it resulted in stuttering, lags, and graphical artifacts (in cases when it worked). Today, even $700 graphics cards are a much better product from a user perspective than the high-end cards of that time (and that's not even taking into account that $700 cards back then were much more expensive).

Yes, this will definitely renew interest in Stadia type products.

Don’t you worry - Microsoft and Amazon will have you covered with cloud streaming.

Can’t afford a computer because they bought up all the supply? They’ll conveniently sell it back to you with a subscription!

You’ll own nothing and be happy.

It's more likely to kill the AI market. They're overbuilding capacity and most of it is going unused. The upcoming haircut is going to kill a lot of the major players.

They've intentionally crafted an unsustainable business model in an effort to get users in the front door and raise their MAUs. We've seen this story before. We should know precisely where it's headed.

I fully agree, the billion dollar question is when it will come.

If it crashes after it kills the PC we’ll be left with… nothing? Path matters as much as destination

Looking at the history of the memory industry the biggest risk is that a firm would over produce and go bankrupt. Maybe this time is different but so far no memory chip maker has gone under because their competition increased capacity.

Is there any indication research is being focused on reducing menory footprint of inference for frontier class models? Is the low hanging fruit already gone there?

What is the risk? Competition is good for consumers.

These are fundamentally different points in design space though, hbm doesn’t have a 10mw idle draw like lpddr does.

I hear people are doing AI workloads on apple hardware, which is LPDDR but with a wider memory bus (1024bit). This requires the SoC to support this; from what I understand not many of any beyond Apple offer this. A wider memory bus may be all we need.

Can’t put HBM in smartphones and laptops. The power drain is too great.

Not many workloads are RAM bandwidth limited. Power and latency are much more common bottlenecks, and HBM loses on both of those.

Oracle is getting sold because of how much capex they're spending on new data centers in the middle of a high rates environment. It's not like they're stockpiling cash due to doubting AI.

> [...] better idea to directly fund the expansion of capacity [...] > > However, that the hyperscalers and AI companies aren't doing this says a lot about their true beliefs about how much future demand AI will have.

With what money? They have to spend the money they get on hardware ASAP else they are left behind.

Intel struggled because they bet the company that Moore's law was over back in ~2014, and instead of upgrading their fabs to EUV they sent the money back to shareholders.

They forgot Moore's main lesson: only the paranoid survive. They thought they could coast, and it nearly killed them.

> Making memory isn’t super hard.

Then why do only 3 companies make it?

//Making memory isn’t super hard. That’s why it is a commodity.

These two aren't related.

Dram is a commodity because the you can replace a chip from hynix with a chip from micron, the have the same behaviour.

And a price competitive Dram isn't easy manufacture, or China would have made it already.

> up and down cycles have bankrupted the vast majority of players in the past

Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity.

The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.

They won’t.

>I for one will only buy Chinese RAM from now on, no matter the prices.

Memory is a commodity, so I think you will be very lonely in your quest.

Possibly the best deal there is

I really need to shut up, or bite the bullet and by one.

If you graph the tokens per second on the 5090, your jaw will hit the floor at how cheap it is

Which surely is the highest it'll ever be! You're suggesting that the price will go down in the future? Would love to hear more about your thought process!

There was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.

I believe msrp is $2000 right?

> The RTX 5090 is faster than an H200. It just has less ram

H200 has HBM and much more 64-bit compute

It makes me sad for the Neo 2.0. More ram is the only thing stopping me switching to it from a Pro.

They’re going to pivot to you renting desktop cloud compute instead of owning anything.

“Bubble”

Because RAM isn’t in PCs only. It’s in tablets, phones, laptops, DIY computers like the Raspberry, mini PCs, watches, smart TVs, game consoles, cars, routers, cameras, all smart appliances from refrigerators to washing machines, fitness trackers, printers etc. Cloud services are irrelevant to most of these categories.

I know the differences between SRAM, DRAM, ...

I asked for evidence different people keep feeding me opposite stories: one insists its not fab capacity but wafer competition, with a recent article claiming HBM3E takes 3 times as much wafer area per bit than LPDDR5X. Others tell me the complete opposite: its fab capacity, not wafer shortage.

Do we have citable references to ground either set of claims?

And that article is contradicting other voices. If that article were correctly identifying the bottleneck as wafer shortage due to switching to HBM, why is everybody discussing the memory makers instead of the boule growers. Memory makers can expand operations all they can, which makes no sense if wafer supply doesn't follow, and the article is suspicously light on semiconductor boule / wafer mfr's.

So which is the bottleneck: fabs or boule growing?

also consider how most solar panels are monocrystalline silicon, how credible is silicon wafer shortage ... really? there is so much disinformation in this market...

Is there an understanding of what OpenAI intends to do with that memory?

Surely they need GPU capacity and would need memory for those GPUs but OpenAI doesn't build GPUs or any hardware, right? So did they pay to keep the supply locked up, or do they have the ability to put that ram into use?

A lot of words to say that Sam Altman bought up the worlds total supply of ram chips for the next few years.

"That leaves small SAAS companies exposed to incoming inflation in the cost of hosting".

Which they will pass on to their customers. If their product provides enough value the customers will pay.....

Capex expenditure start exploding after covid with the chart going hockey stick at the end of 23/start of 24, almost 2.5 years ago.

A lot of capex is supposed to go into the datacentres, didn't they know that datacentres need to be filled among other stuff with RAM? I wonder if at some point we will discover that there is a shortage of fibre optic cables of SFPs ...

PS: Obviously armchair economist here too ... but for it doesn't seem too difficult to foresee the increase of the demand.

As Yogi Berra famously said, "It's tough to make predictions, especially about the future." But based on historical tech industry trends, a price increase in one component that's this rapid and extreme, is likely to eventually regress somewhat toward the long-term trend line - even if that trend line experiences a longer-term shift upward.

As always, some interpret certain recent events as reason to conclude "but this time it's different." Occasionally they are correct. But that doesn't change the fact that it's reasonable to assume some of the recent extreme, rapid price inflation is due to shorter term market distortion. It's also pretty clear that some of the recent increase in demand represents a stable increase in the long-term trendline. The question is how much is long-term stable and how much is short-term distortion.

Then I will make my build last as long as it can, in protest of that. I do expect at least a performative price drop in the coming years, though.

Then I better divert all of my investment into memory maker stocks.

You could make the same economic argument for any SaaS, but the margins SaaS providers look for make it so that the only time it isn't cheaper to run your own software/hardware stack in place of SaaS is when the hardware requirements are very low, not high. SaaS makes sense economically when you take into account the admin, compliance, etc. costs... and the admin costs of a Nintendo Switch are pretty low.

Economic efficiency does not win the day because the free market is a myth. Cloud gaming is a technically worse solution because the latency floor is higher. It's a microeconomic disaster (rent vs buy, buy wins). The only reason it would become a thing is if the multinationals succeed in concentrating more wealth and power, which consumers aren't interested in supporting. It's a bad deal and consumers know it. They would have to be forced into it by having the consumer hardware market taken off the table (which is happening and the only possible avenue for a technical regression like cloud gaming to have a market).

If DLSS 5 becomes the norm it's possible that just makes things worse. The DLSS 5 demos required an entire separate card to run the model, though IIRC NVIDIA did claim it would eventually work on a single card. Given what the model is doing (yassifying the whole scene instead of just upscaling/reconstructing) it makes sense to me that it would increase compute demand instead of reduce it like previous versions of DLSS.

From my point of view, I suppose we will enter a "Let AI generate entertainment" era. In which you just might rent everything, including games. No need for a beefy computer at home, you just need a slim endpoint:

"Order yours now, for just $99.99 per month, hardware included! Order today, and you will get three months of 'Office Suite' for free, with a small additional cost of $49.99 after month 4. On a tight budget? Switch to the yearly subscription, and pay comfortably in 18 installments."

Every time we add capacity with almost anything, we find ways to saturate it.