In a groundbreaking move, Microsoft has announced the development of its custom AI chip, the Azure Maia, and an Arm-based CPU named Azure Cobalt. These chips are strategically designed to power Microsoft’s Azure data centers, ushering in a new era of cloud infrastructure optimized for artificial intelligence (AI) workloads. The move comes in response to the surging demand for Nvidia’s H100 GPUs, which are widely used for training large language models and generative image tools.
Microsoft’s venture into silicon development is not new, with a history dating back to collaborating on the Xbox’s chip over two decades ago. The company has also co-engineered chips for its Surface devices. The Azure Maia and Azure Cobalt represent the culmination of Microsoft’s extensive experience in silicon development, stemming from a strategic effort that began in 2017 to architect a cloud hardware stack.
Azure Cobalt CPU
The Azure Cobalt CPU is a 128-core chip built on an Arm Neoverse CSS design, tailored for Microsoft’s cloud services on Azure. With a focus on performance, power management, and intentional design choices, the Azure Cobalt CPU aims to outperform existing Arm-based servers in Microsoft’s data centers. Initial testing indicates a performance boost of up to 40% over current commercial Arm servers.
Azure Maia AI Chip
The Azure Maia 100 AI accelerator, designed for cloud AI workloads, boasts 105 billion transistors manufactured on a 5-nanometer TSMC process. With features like support for sub 8-bit data types (MX data types), Maia is optimized for faster model training and inference times. Microsoft is testing Maia on GPT 3.5 Turbo, the model powering ChatGPT, Bing AI workloads, and GitHub Copilot.
Microsoft’s partnership with OpenAI involves collaborative design and testing of the Maia chip, showcasing its commitment to advancing AI capabilities. Microsoft is also part of a group, including AMD, Arm, Intel, Meta, Nvidia, and Qualcomm, standardizing the next generation of data formats for AI models. This collaborative approach aims to adapt entire systems to meet the evolving needs of AI.
Unique Liquid-Cooled Design and Rack Sharing
Maia introduces Microsoft’s first complete liquid-cooled server processor, enabling higher server density and efficiency within the existing data center footprint. Microsoft is sharing its rack designs, including MX data types, with partners for use in systems with different silicon. However, the Maia chip designs remain proprietary to Microsoft.
Future Prospects and Diversification
The naming convention (Maia 100 and Cobalt 100) suggests Microsoft’s intention to develop second-generation versions of these chips. While Microsoft emphasizes the importance of partnerships with existing suppliers like Nvidia and AMD, the in-house development of chips offers diversification in the supply chain, potentially reducing the cost of AI for customers.
Conclusion
Microsoft’s Azure Maia AI chip and Azure Cobalt CPU represent a significant leap forward in the company’s pursuit of AI-optimized cloud infrastructure. The collaboration with OpenAI, commitment to industry standardization, and innovative design choices underscore Microsoft’s dedication to shaping the future of AI. As these chips are deployed, the impact on AI cloud services pricing and the realization of Microsoft’s broader AI ambitions will be closely watched.
