Microsoft Sees Composable Memory as a Future Direction for AI

Microsoft has identified memory as a constraint on AI inference, while principal architect Samir Rajadnya described composable architecture as a possible direction over the next several years, not a new Azure deployment.

Composable memory separates capacity from individual servers so systems can allocate it to workloads as needed. Microsoft has not announced a composable-memory deployment for Azure AI workloads.

Microsoft has said AI inference performance is limited by the memory needed to keep larger models and longer contexts available to computing systems. Its stated work spans model changes, software management, silicon efficiency and compiler placement.

Microsoft’s Pond research studied CXL-based memory pooling using Azure production traces. In the scenarios analyzed, up to 25% of DRAM could be stranded. An evaluation of 158 workloads found a 7% reduction in DRAM costs, with performance within 1% to 5% of virtual machines using memory on the same NUMA node.

In 2025, Microsoft announced a private preview of Compute Express Link technology for Azure M-series virtual machines in collaboration with SAP and Intel. That effort focused on memory configurations for SAP workloads; it does not establish a broad pooled-memory deployment for AI workloads.