As artificial intelligence workloads continue to scale, the semiconductor industry is preparing for a post-HBM era. High-Bandwidth Memory (HBM) has become a cornerstone of AI accelerators thanks to its stacked DRAM architecture and massive bandwidth, but HBM alone is no longer sufficient. Power consumption, capacity limits, and cost pressures are forcing memory vendors and system designers to explore new architectures that can move and store far larger datasets more efficiently.
The next phase of innovation is not about replacing HBM outright, but about complementing and extending it with new memory form factors, interconnects, and hybrid designs.
🔋 Low-Power, High-Capacity Memory Modules #
One of the most prominent candidates for next-generation AI systems is SOCAMM (Small Outline Compression Attached Memory Module).
SOCAMM is a memory module designed specifically for AI servers, built on low-power DRAM (LPDDR) rather than traditional server DDR. By combining multiple LPDDR devices into a compact module, SOCAMM delivers significantly higher power efficiency, addressing one of the most critical bottlenecks in modern AI data centers.
NVIDIA is widely expected to adopt SOCAMM in its next-generation AI accelerator platform, Rubin, signaling strong ecosystem confidence. Memory vendors are responding quickly:
- Micron has unveiled SOCAMM2, claiming a 20% improvement in power efficiency over earlier designs.
- Samsung Electronics and SK Hynix are actively developing SOCAMM variants with further reductions in power consumption and higher effective bandwidth.
In the AI era, performance alone is no longer enough—energy efficiency increasingly determines whether a technology can be deployed at scale.
đź”— CXL: Breaking the Memory Capacity Wall #
Another major pillar of post-HBM innovation is Compute Express Link (CXL). Traditional system architectures bind memory directly to CPUs or GPUs, creating rigid capacity limits and inefficient resource utilization. CXL changes this model.
CXL enables memory pooling, allowing large shared memory resources to be dynamically attached to CPUs or GPUs as needed. In theory, this allows near-unlimited memory expansion and far more flexible allocation across workloads.
Industry progress is accelerating:
- Samsung Electronics has completed mass-production readiness for CXL 2.0-based DRAM.
- SK Hynix has developed a CXL 2.0 DRAM solution offering 50% greater capacity than conventional DDR5 modules.
- Startups such as Panmnesia and Primemass are advancing CXL controllers and switching technologies to support large-scale deployments.
For AI training and inference, where dataset sizes can exceed local memory limits, CXL is emerging as a foundational technology rather than an optional enhancement.
🧠HBF and HBS: Expanding Beyond DRAM #
While HBM remains indispensable for high-speed computation, researchers and vendors are exploring alternatives to extend memory capacity further.
High-Bandwidth Flash (HBF) #
HBF (High-Bandwidth Flash) replaces stacked DRAM with stacked NAND flash. While DRAM functions as a fast “workbench,” NAND serves as a high-density “warehouse,” retaining data even without power.
HBF is designed for AI workloads that require massive datasets and high-throughput read/write operations, rather than ultra-low latency alone. Key characteristics include:
- Higher achievable stack heights than HBM
- Optimizations for large-scale data access
- Lower cost per bit compared to DRAM-based solutions
According to industry forecasts, HBF could begin commercialization around 2027, with the market reaching approximately $12 billion by 2030. Experts emphasize that HBF is not a replacement for HBM, but a complementary technology that extends system memory hierarchies.
High-Bandwidth Storage (HBS) #
An even more ambitious concept is High-Bandwidth Storage (HBS), which integrates DRAM and NAND within a single package. This hybrid approach aims to combine DRAM’s speed with NAND’s capacity.
SK Hynix is reportedly exploring HBS for mobile and edge applications, where space and power constraints are especially severe.
⚙️ Processing-in-Memory: Collapsing the Compute–Memory Gap #
Beyond new memory types, the industry is also rethinking the fundamental separation between computation and storage.
Processing-in-Memory (PIM) embeds compute logic directly into memory devices, allowing data to be processed where it is stored. This reduces data movement, lowers power consumption, and can dramatically improve AI efficiency.
- Samsung Electronics is pursuing PIM as a core strategy for future AI accelerators.
- SK Hynix is developing LPDDR6-based PIM, targeting power-efficient AI and mobile workloads.
As one industry insider summarized, the boundary between memory and logic is dissolving. In the AI era, performance gains increasingly come from architectural integration rather than raw transistor scaling.
đź“Ś Conclusion #
The post-HBM roadmap is not defined by a single breakthrough, but by a convergence of technologies: SOCAMM for power efficiency, CXL for scalability, HBF and HBS for capacity expansion, and PIM for architectural efficiency. Together, these innovations signal a future where memory is no longer a passive component, but an active, flexible participant in AI computation.
HBM remains critical—but the next decade of AI will be built on what comes alongside it.