Microsoft unveils Maia 300, its third-generation AI inference chip

Microsoft details Maia 300, an in-house accelerator for large language model inference on Azure with 288 GB of HBM4 per chip, up to 7x the rack-scale token throughput of Maia 200 and more than 2.5x the performance per watt. Superpods scale up to 12,288 accelerators.