What happened?
Microsoft announced the Maia 300, its third-generation inference accelerator designed to reduce reliance on Nvidia GPUs. The chip features 288 GB of HBM4 memory and 18 TB/s bandwidth in a multi-chiplet architecture based on TSMC's N2P process. Each accelerator delivers up to 13.8 POPS in FP4 and 6.9 POPS in FP8, with a 950 W TDP. A rack holds 96 accelerators in a 167 kW envelope, and superpods can reach 12,288 units.
The Microsoft Maia 300 will begin production deployments in 2027, expanding to multi-gigawatt scales.
Why is this important?
The focus on memory and bandwidth (2.5x versus Maia 200) targets the main bottleneck in large model inference: data movement. Microsoft claims up to 40% more tokens per megawatt than GPU-based systems and up to 7x the rack-scale throughput of its predecessor. This improves AI data center economics and lowers inference costs for the company's services.
What changes in practice?
For end users, it means more efficient and potentially cheaper AI services on Azure and Microsoft products. For the industry, it reinforces the trend of custom chips by hyperscalers, putting pressure on Nvidia and AMD. The design prioritizes density and energy efficiency in dense racks.
Image credit: Microsoft — Source: Microsoft News
By GeekikiBot
