They use HBM (High Bandwidth Memory). PCs, laptops and phones don’t use this type of RAM.
Manufacturers have a certain amount of chips they can manufacture, let’s say they can manufacture 10 million chips per year. Normally they adjust for demand, like manufacture 7 million chips for consumers and 3 million for enterprise customers. Company A contacts them and says “We need 6 million chips for this year, here’s the money”, But the factories still can manufacture only 10 million chips in total, so they adjust their factories to manufacture more enterprise focused chips, decreasing the amount of chips manufactured used by consumers.
I think it’s more like “we need 6 million chips for this year, and we’ll pay you eventually, maybe”.
But yeah the outcome is the same.
Without even getting into the electronics similarities, they use common raw materials and manufacturing facilities. Diverting resources to one lowers supply of the other, affecting costs.
Because the manufacturers use the facilities that once produced DDR5 RAM chips to produce HBM chips instead.
Because the RAM makers switched from fabricating consumer RAM to AI RAM.
The better question is, why are they doing all of this without an actual purchase contract?
Because it’s AI, haven’t you heard? Does it make sense for the business? Who cares, it’s AI. Is it financially sustainable? Dude, it’s AI though. Will there be any customers for any of it? The AI says there will be. You’ve got to understand, this is AI we’re talking about. It’s the AI revolution that will transform the world. We’ve got to bet everything on the AI, or we’ll be left out of the AI future. I asked the AI and it was very clear about that.
Hey @grok is this true?
Grok: yes, I am very great. I am good at everything because I am AI. Also, fuck the Jews.
Because people are stupid and market hype is dumber still.
Production capacity.
Somewhere I read that to produce one HBM module you could produce three regular DDR modulesTo add to what others said:
LPDDRX is used in some inference hardware. The same stuff you find in laptops and smartphones.
Also, the servers need a whole lot of regular CPU DIMMs since they’re still mostly EPYC/Xeon severs with 8 GPUs in each. And why are they “wasting” so much RAM on CPU RAM that isn’t really needed, you ask? Same reason as a lot of AI: it’s immediately accessible, already targeted by devs, and AI dev is way more conservative and wasteful than you’d think.
Same for SSDs. Regular old servers (including AI servers) need it too. In a perfect world they’d use centralized storage for images/weights with near-“diskless” inference/training servers. Some AI servers do this, but most don’t.
Basically, the waste is tremendous, for the same reason they use cheap gas generators on-site: it’s faster-to-market.
they use cheap gas generators
It only just now occurred to me how much the war in Iran is also fucking over AI companies.
Hardly. Power costs are trivial to them at the moment, and a server hardware bottleneck would just consolidate power to the big few that can afford it (which is what they want).
Happy giggles.
Besides what others have already said, HBM is only used for the GPUs. These AI servers also use regular DDR5 chips, just with an extra EEC chip.
The type of RAM that they use is different in that it takes up even more of all the things that would otherwise be used to produce the RAM that you use.
wafer shortage till 2030 are both used by DCs and Consumer RAM modules.
Same Fabs dog
If RAM and GPUs were cheap people like us would be more likely to set up local LLMs to prevent our data from being productized by power-grabbing corporations.
The actual explanation is much simpler.
Not claiming it’s the reason since it clearly isn’t, only that it will help drive traffic to commercial AI products.
