Microsoft published the architecture of Maia 300 on October 12, the third generation of its in-house AI accelerator, and says it delivers up to 7 times the rack-scale token throughput of Maia 200 with more than 2.5 times the performance per watt. The chip is built for agentic inference, the multi-step, long-context workloads that now run behind Copilot and Foundry.
What Microsoft disclosed
According to the Azure Hardware Systems team’s architecture post, each Maia 300 accelerator carries eight HBM4 memory stacks with 288 GB of capacity and 18 TB/s of bandwidth. Against Maia 200 that is 2.5 times the memory bandwidth, 33% more capacity and roughly twice the HBM bandwidth per watt. Compute sits on TSMC’s N2P process, and Microsoft says the multi-chiplet design raises compute FLOPs by 36% and scale-up bandwidth by 14%.
The rack doubles to 96 accelerators in 24 trays inside a 167 kW envelope. Six switch ASICs in three liquid-cooled trays connect the rack, and liquid cooling removes more than 80% of the thermal load. A second, optical tier links up to 128 racks into a superpod of 12,288 accelerators. Microsoft describes one fabric across the pod and the superpod, with prefill and decode workers sized and placed independently, and with hot-spare capacity in a neighboring rack covering failed trays.
The post does not give a ship date. The Decoder, summarizing a separate Microsoft announcement by executive Scott Guthrie, reports that Maia 300 ships in 2027, that Microsoft plans multi-gigawatt deployments from that year, and that Maia 200 already runs Microsoft Copilot in the US, Europe and Australia, with GitHub Copilot and Foundry following by year end. We could not retrieve Guthrie’s own post, so those details rest on The Decoder’s account.
What the two headline figures imply together
Microsoft gives two generation-over-generation numbers: up to 7 times the rack throughput and more than 2.5 times the performance per watt. If both describe the same workload, dividing one by the other gives about 2.8 times the power per rack. Microsoft states no such figure, so this is our own arithmetic. It would put a Maia 200 rack near 60 kW against the 167 kW stated for Maia 300. Microsoft does not publish the Maia 200 rack figure in this post, so the check cannot be completed from the source.
The gain comes from density and system design, with efficiency a smaller part of it. The 7 times figure is for the full 96-accelerator configuration, twice the accelerators per rack of the prior generation. A buyer reading “7 times” as a per-chip or per-dollar improvement would overstate it, because the rack holds twice as many accelerators as before. The post states its goal as useful tokens per watt and per dollar, but it gives no per-dollar result.
Where the claims stop
The architecture post compares Maia 300 only with Maia 200. The comparison with GPUs appears in the secondary account: The Decoder reports Microsoft’s claims of up to 1.7 times more tokens per watt than leading GPUs for Teams on Maia 200, and 40% more tokens per megawatt than leading GPU systems for Maia 300. The reporting names no reference system, model or batch size. Microsoft says more detail is coming at Ignite in November, and until then the GPU comparisons are vendor claims without a published baseline.
Microsoft is not alone in betting that inference deserves its own silicon. Our coverage of Etched’s valuation jump and of Anthropic’s own chip effort shows the same direction from a startup and from a model lab.
What it means for the buying committee
Most enterprises will never buy a Maia accelerator. They buy Copilot seats, Foundry capacity and Azure OpenAI throughput, and the silicon underneath changes what Microsoft pays to serve a token. Whether that saving reaches a customer’s invoice is a contract question that the post does not touch. Gartner’s forecast that AI infrastructure drives 2026 IT spending explains why every hyperscaler is trying to lower its serving cost, and also why buyers should expect to be asked to commit to capacity before those savings are visible.
The Decoder reports that Maia serves OpenAI models, Microsoft’s own MAI models and open-source models. A customer whose workload depends on a third-party model should ask which of those models run on Maia, in which regions, and whether the same model on a GPU fleet is priced and performs identically. The answer decides whether the silicon is invisible to you or a variable in your latency and cost forecast.
Questions to put to the account team
CIOs and heads of AI platform with committed Azure spend can use the October disclosure to ask for four items in writing before Ignite. First, whether any Foundry or Copilot service-level commitment changes when a workload moves from GPU to Maia hardware. Second, whether unit pricing for the same model differs by silicon. Third, which regions get Maia capacity first, since each 167 kW rack connects to facility water or a heat-exchanger unit and site readiness varies. Fourth, what happens to a committed-spend agreement if Microsoft shifts a service to Maia mid-term.
Capacity planners should also note the timing. Maia 300 shipping in 2027, if The Decoder has it right, means the chip does not change any 2026 forecast. It does belong in the 2027 renewal conversation, where the efficiency gain Microsoft is claiming is the lever a customer can reasonably ask to share.
Source: Microsoft Azure Blog
