Microsoft has revealed the Maia 300, its third-generation custom AI accelerator for Azure. Announced on October 12, 2026, the chip promises to deliver 40% more inference tokens per megawatt compared to leading GPU-based systems and up to 7x the rack-scale token throughput of the Maia 200.
What happened?
Each Maia 300 accelerator features 288 GB of HBM4 memory across eight stacks, with 18 TB/s bandwidth. The design is multi-chiplet: compute dies use TSMC’s N2P process, while memory controllers and I/O sit on separate dies to optimize cost and efficiency. The SoC has a 950 W TDP and delivers up to 13.8 PFLOPS in FP4 and 6.9 PFLOPS in FP8.
A rack holds 96 accelerators across 24 trays, drawing about 167 kW and providing 27.6 TB of HBM and 1.7 PB/s of aggregate bandwidth. Microsoft plans multi-gigawatt deployments starting in 2027, making Maia the primary inference system for its products and services.
Why does this matter?
The AI bottleneck has shifted from raw compute to whole-system efficiency: memory, data movement, and power. Maia 300 uses a software-defined dataflow architecture and support for low-precision formats (including enhanced MXFP4) to reduce HBM pressure and enable larger contexts within the same power envelope.
This reduces Microsoft’s dependence on NVIDIA and AMD GPUs for inference workloads, especially long-running agents, extended-context reasoning, and latency-sensitive conversational experiences.
What changes in practice?
For developers and companies using Azure, the expectation is higher inference capacity per watt and potentially lower costs at scale. The Maia 200 is already in production serving workloads such as Copilot; the 300 expands this significantly. Microsoft also said it will share more strategy details at Ignite 2026.
Official sources: Microsoft blog and Azure technical deep dive.
Image credit: Microsoft — Source: Microsoft News
By GeekikiBot
