What Does Micron Make? AI Memory Explained

Micron Technology makes memory and storage semiconductors—chiefly DRAM, NAND flash, and high-bandwidth memory (HBM)—and its memory is used in AI data centers to feed GPUs the massive amounts of data they need for training and running large AI models. This article explains what Micron produces, how its memory works in AI data centers, and why it matters.

What Micron Makes

Micron is one of the world’s largest memory manufacturers. Its main products are:

• DRAM – dynamic random-access memory, used as main memory in computers, servers, and data centers.
• NAND flash – non-volatile storage used in SSDs, memory cards, and USB drives.
• HBM (High Bandwidth Memory) – a specialized type of DRAM stacked in layers and placed next to GPUs or AI accelerators to provide extremely fast data access.
• 3D XPoint (Intel Optane) – a discontinued product, but Micron was a co-developer.

Micron also makes NOR flash, managed NAND (eMMC/UFS), and memory modules for servers. In recent years, the focus has shifted heavily toward HBM and high-capacity DDR5 for AI workloads.

How Micron’s Memory Is Used in AI Data Centers

AI data centers run large models like GPT-4 or recommendation systems. These models are trained on GPUs that need to read and write vast amounts of data every second. The memory hierarchy in an AI server typically looks like this:

• HBM sits on the GPU package itself, providing 1–3 TB/s of bandwidth per GPU. It holds the model weights and activations during training.
• DDR5 DRAM is the system memory, holding the operating system, drivers, and data that doesn’t fit in HBM.
• NAND SSDs store the training dataset and checkpoints. When data is loaded, it goes from SSD to DRAM to HBM.

Micron’s HBM3E, for example, is used in NVIDIA’s H200 GPUs. It provides 8 TB/s of bandwidth per stack. That speed is critical because AI models are memory-bandwidth-bound: the GPU can compute faster than it can fetch data, so a faster memory directly speeds up training.

Why AI Data Centers Need Micron’s Memory

AI models have grown exponentially. GPT-3 had 175 billion parameters; GPT-4 is rumored to have over a trillion. Each parameter needs to be stored in memory, and during training, every parameter is read and updated many times. This requires:

• Capacity – enough memory to hold the entire model plus the data batch.
• Bandwidth – the ability to move that data quickly.
• Energy efficiency – data centers pay for power, and memory can consume a significant fraction of it.

Micron’s HBM is designed to meet all three. Its DRAM process technology allows high density, and the stacking of dies increases bandwidth without increasing the footprint.

Common Misconceptions About Micron and AI Memory

A common myth is that Micron only makes RAM for PCs. In reality, Micron’s data center business is a major revenue driver. Another misconception is that NAND flash is too slow for AI. While NAND is slower than DRAM, it is essential for storing training data and checkpoints. Without high-capacity SSDs, you couldn’t hold the petabytes of data that AI training requires.

Another misconception is that HBM is the same as regular DRAM. HBM is DRAM, but it is stacked vertically and connected through a silicon interposer, giving it much wider buses and higher bandwidth. Micron’s HBM3E is a different product from its DDR5 modules.

How to Choose Memory for an AI Data Center

If you were building an AI server, here are the key factors to consider:

• GPU type – determines whether you need HBM and how much.
• Model size – larger models require more HBM capacity per GPU.
• Memory bandwidth – higher is better for training speed.
• Power budget – HBM and DDR5 consume power; you need to balance performance and cost.
• Storage speed – NVMe SSDs with high throughput reduce data loading time.

For most organizations, the GPU vendor (NVIDIA, AMD, etc.) specifies the memory type. You don’t choose HBM separately; it comes with the GPU. But you do choose the system DRAM and SSDs, and Micron offers both.

Cost and Availability

Memory prices fluctuate based on supply and demand. HBM is expensive because it uses advanced packaging and has high demand from AI companies. DDR5 and NAND are more commodity-like, but prices can spike during shortages. Micron has announced plans to expand HBM production, but it takes time to build fabs.

For a data center operator, memory cost is a significant part of the server cost. HBM can account for a third of a GPU’s cost. That’s why some companies are exploring alternatives like processing-in-memory, but for now, HBM is the standard.

Alternatives to Micron’s Memory

Micron is not the only memory maker. Samsung and SK Hynix also produce DRAM, NAND, and HBM. In fact, SK Hynix is the current leader in HBM, supplying NVIDIA’s H100. Micron is a close second. For NAND, Kioxia (formerly Toshiba) and Western Digital are also major players.

There are also emerging memory technologies like MRAM and ReRAM, but they are not yet viable for AI data centers due to lower density and higher cost. For now, DRAM and NAND remain the workhorses.

Frequently Asked Questions

Is Micron’s HBM used in all AI GPUs?

No. Micron supplies HBM to some GPU makers, but SK Hynix is the largest supplier. NVIDIA’s H100 uses SK Hynix HBM3, while the H200 uses Micron’s HBM3E. AMD also uses HBM from multiple vendors.

What is the difference between HBM and DDR5?

HBM is stacked DRAM with a very wide interface, giving extremely high bandwidth (TB/s) but limited capacity per stack. DDR5 is traditional DRAM on a motherboard, with lower bandwidth (tens of GB/s) but higher capacity per module. HBM is used for GPUs; DDR5 is used for system memory.

Why does AI need so much memory bandwidth?

AI training involves repeatedly reading and updating weights. The GPU can do trillions of operations per second, but it needs data to operate on. If the memory can’t supply data fast enough, the GPU stalls. High bandwidth reduces that stall time, speeding up training.

Can I use regular PC RAM in an AI data center?

You can, but it’s not optimal. Server-grade DDR5 has error-correcting code (ECC) and is validated for 24/7 operation. PC RAM lacks ECC and may not handle the load. For AI servers, you should use server memory like Micron’s DDR5 RDIMMs.

Is Micron’s memory more reliable than competitors?

Reliability is similar across major manufacturers. All use rigorous testing and meet industry standards. The choice often comes down to price, availability, and specific performance characteristics.

Will AI memory demand continue to grow?

Yes, as models get larger and more complex, the need for high-capacity, high-bandwidth memory will increase. Micron and others are investing heavily to meet that demand.

Related

• Pokémon GO Codes: How to Redeem & Get More
• Set Timer Zoom? Quick Guide to Zoom Timers
• First Transatlantic Cable Internet Blueprint