Microsoft Scales Up AI Infrastructure with Custom Clea Maia 300 Accelerators to Tackle the Global Inference Bottleneck

As the generative AI revolution matures, the industry has pivoted from the initial scramble for raw training power to a more complex and economically pressing challenge: the inference problem. Even with the current generation of state-of-the-art GPU and XPU accelerators, the cost of generating tokens remains high, creating a significant barrier to the widespread, sustainable deployment of large language models. For Microsoft, which operates as both a primary host for OpenAI’s massive ChatGPT models and a provider of AI-infused productivity tools for millions of enterprise users, this inference problem is perhaps the most acute in the world.

The scale of Microsoft’s internal demand is staggering. Beyond the massive compute requirements of training and running OpenAI’s foundation models, Microsoft is rapidly integrating AI routines into every corner of its worker productivity and back-office applications. This integration is driving usage up a "hockey stick" growth curve, necessitating a hardware strategy that moves faster than the traditional commercial silicon cycle. To meet this demand, Microsoft is accelerating the development of its homegrown Maia family of custom XPU accelerators. At this week’s Open Compute Project (OCP) Summit 2026, the company provided a detailed look at the "Clea" Maia 300, a third-generation accelerator designed to dramatically expand the scale and efficiency of AI inference on the Azure cloud.

The Maia 300 represents a significant architectural leap over its predecessors, specifically tailored to handle the emerging complexity of "agentic" workloads. These workloads often involve dozens or even hundreds of models working in concert—whether through a Mixture of Experts (MoE) foundation model architecture, physically distinct independent models, or a hybrid of both. For these systems to function with low latency, they require a massive, shared network and memory space, a requirement that the Maia 300 is built to address directly.

Microsoft’s journey into custom silicon has been a rapid evolution. The company first entered the space with the "Athena" Maia 100 in November 2023, alongside its custom Arm-based Cobalt 100 server CPU. While the Maia 100 served as a foundational step, its successor, the "Braga" Maia 200—unveiled only in January of this year—marked a more serious entry into production. The Maia 200 has been ramping up within Azure data centers throughout 2026, already handling inference for Copilot and supporting the development of new models. According to Scott Guthrie, Executive Vice President of Cloud and AI at Microsoft, the Azure Maia program has become a cornerstone of the company’s infrastructure strategy. By the end of this year, GitHub Copilot and the Microsoft Foundry are expected to transition their workloads entirely to Maia 200 systems, which also support various open-source and proprietary OpenAI models.

The strategic ambition behind the Maia program is reflected in its scale. Industry reports suggest Microsoft aims to deploy at least one million Maia sockets in the field. Guthrie has noted that Maia will eventually support "multi-gigawatt" deployments on Azure. To put that in perspective, a single gigawatt of Maia 300 capacity would encompass roughly 575,000 devices, considering the power draw of the accelerator alongside its associated host, storage, and networking components. Microsoft has disclosed that a single rack of Maia 300s, containing 96 interlinked XPUs, consumes approximately 167 kilowatts.