# SK hynix and Peers Show AI Memory Designs Beyond HBM at OCP

By Simon Yoon

Canonical URL: https://www.tokenpost.com/news/technology/56763
Published: 2026-11-13T14:29:02.000Z
Updated: 2026-11-13T14:29:02.000Z
Section: Technology

> SK hynix's CXL modules had 80 GB/s of bandwidth, while a shared SSD pool demonstrated large gains for AI cache workloads.

Hardware companies at the Open Compute Project Global Summit showed ways to extend AI memory beyond processor-attached high-bandwidth memory (HBM), addressing cache growth in multi-step workloads.

The summit took place in San Jose from Oct. 12-15. The presentations addressed agentic AI systems that handle multi-step tasks and extended exchanges. Each conversational turn adds to the key-value (KV) cache.

SK hynix presented its Memory-centric AI Rack System (MARS) alongside 256 GB CMM-DDR5 modules based on Compute Express Link (CXL) 3.2. CXL allows processors to access memory outside their own package. SK hynix listed the modules' bandwidth at 80 GB/s and said they were about 30% more energy-efficient than previous generations.

Metrum AI and Solidigm demonstrated a shared solid-state drive (SSD) pool for KV-cache data, shifting it from more expensive memory into storage. The companies reported up to 64.8 times faster time to first token, the time before a model begins responding, and 40.7 times higher throughput. They also reported 26.5 times lower energy use per GPU request than traditional methods.

Nallasway proposed connecting external DRAM directly to XPUs, a broad term for AI processors. Its module would provide 3.2 TB of LPDDR5X memory.

The presentations covered several approaches alongside HBM: CXL-based memory, external DRAM and SSD storage. Together, they described tiers with different speeds and costs. The SSD demonstration moved cache out of more expensive memory, while the other designs added or connected DRAM beyond the processor package.

The designs were presented as proposals and demonstrations, not confirmed deployments. Their use across memory tiers also depends on software placing data in the appropriate tier and deciding when to move it.