At Hot Chips 2026, SK hynix explained how High Bandwidth Memory is evolving for AI. Future gains will require more than better DRAM circuits. They will also depend on advanced packaging, denser vertical integration, improved thermal management, and closer cooperation between memory and processor teams.
What Is HBM, and How Does It Outperform Traditional Memory?
HBM stacks multiple DRAM dies on top of a base die. Tiny through-silicon vias (TSVs) connect the dies. The package sits next to a GPU or AI accelerator on a silicon interposer. Thousands of parallel connections move data between them.
This wide interface provides much higher bandwidth than conventional memory. It also uses less space on the circuit board. Four HBM3E packages provide 144 gigabytes of capacity and about 4 terabytes per second of bandwidth. A comparable setup with 12 GDDR6 devices uses twice as much board area. HBM3E also uses less energy per transferred bit.
How HBM Meets the Three Core Needs of AI Systems
AI systems need memory with three qualities: high bandwidth, large capacity, and low power use. Each HBM generation improves these areas:
- HBM2E: up to 460 gigabytes per second
- HBM3: up to 717 gigabytes per second
- HBM3E: up to 1,024 gigabytes per second
- HBM4: up to 2,048 gigabytes per second, with the interface expanding from 1,024 to 2,048 connections
Manufacturing Challenges for Next-Gen HBM Stacks
Higher performance makes manufacturing more difficult. HBM4 has more than 20,000 TSVs and about 16,148 micro-bumps. Its larger and taller package creates new challenges. These include wafer thinning, die warpage, bump uniformity, narrow-gap filling, joint reliability, and heat removal.
One defect can ruin the entire stack. Precise inspection is therefore essential. Manufacturers must also test stacked dies to ensure that they are known-good before assembly.
Key Production Technologies Powering HBM Innovation
MR-MUF for High-Volume HBM3E Production
SK hynix uses mass reflow with molded underfill (MR-MUF) for current HBM stacks. Thermo-compression bonding attaches dies one at a time. MR-MUF bonds all connections in a single reflow step and then fills the stack with protective material.
This process improves throughput and lowers thermal resistance. It also requires careful control of warpage and narrow gaps. With advanced MR-MUF, SK hynix built a 16-layer HBM3E stack with 48 gigabytes of capacity. The package height stayed below 775 micrometers.
Hybrid Bonding for 20+ Die HBM Stacks
Hybrid bonding is a promising option for future stacks with 20 or more dies. It directly joins dielectric surfaces and copper connections. This removes the need for solder micro-bumps.
The connections can therefore be much smaller. This leaves room for more TSVs and higher bandwidth. It also creates space for thicker, stronger DRAM dies. Direct copper connections can improve heat flow as the stack grows taller.
Thermal Management: Solving Heat Challenges for Taller HBM Stacks
Thermal management is as important as manufacturing. Higher bandwidth increases power use. Taller stacks also make it harder for heat to escape.
SK hynix’s proposed i-HBM design adds a thermally conductive but electrically insulating material between hot die-to-die interfaces. This creates a separate path for heat. The design could reduce thermal resistance by more than 30%.
Other improvements include advanced logic processes for the base die. These processes can reduce power use. Power TSVs can also be distributed across the package to improve power delivery.
Why This Matters for the Future of AI
AI performance increasingly depends on fast data transfer. An expensive AI accelerator wastes resources when it waits for memory. HBM reduces this bottleneck. Its future performance, however, will depend heavily on advanced packaging.
Bottom Line: Cross-Team Collaboration Is Now Essential
HBM is no longer added near the end of system assembly. It is integrated early with processors and interposers. The memory must also work with technologies such as TSMC CoWoS, redistribution-layer interposers, and embedded bridges.
Memory suppliers, foundries, processor designers, and packaging teams must work together from the start. Advanced packaging now determines whether future AI systems can meet their targets for bandwidth, capacity, reliability, and energy efficiency.
发表回复
要发表评论，您必须先登录。