Microsoft now has its own customized AI chip. As well as reducing its dependence on Nvidia, the company will be strengthening its cloud infrastructure.
The U.S. computer giant is thus getting a custom chip tailored to the needs of large language models. It took advantage of its annual developer conference this week to present Maia 100. Microsoft also unveiled its AI chipits new processor Cobalt 100.
Artificial intelligence makes our daily lives much easier. What we don’t realize is the burden it places on the technology companies that provide these services. These companies, which include Microsoft, face exorbitant costs. These solutions are ten times more expensive to provide than conventional IT services.
Demand for services that include artificial intelligence is rising sharply. This growth accentuates the risk of component shortages. Indeed, the main supplier, Nvidiamay not be able to keep up. This context has motivated the American computer giant to develop its own in-house solution.
Microsoft will not market its AI chip
Maia 100’s design takes into account the execution of large language models such as GPT 3.5 Turbo and GPT-4. These two LLMs are at the heart of the Microsoft Azure and Copilot.
For your information, the new AI chip has 105 billion transistorswhich were manufactured using 5 nm TSMC process. In addition, the Cobalt 100 processor is based on a 128-core ARM structure.
It should be noted that the American computer giant has no plans to sell its first AI chip and its new processor. It prefers to keep them for exclusive internal use. On the other hand, the company will use its two new components to equip its data centers next year.
In fact, the components will be mounted on customized server boards and installed in custom-made racks. Microsoft says it has optimized its hardware to work with its software suites. This will boost performance and unlock new capabilities.
Combining internal and external solutions
Although Microsoft now has its own AI chip, the company has no plans to discontinue the use of third-party components. It plans to continue sourcing from its partners, both to meet its procurement needs. The aim is also to satisfy its employees.
Note that next year, the GPU H200 Tensor Core from Nvidia will be included in the data center fleet. The graphics processor will support inference on larger LLMs without increasing latency. Bill Gates’ company will also be relying on the super-powerful MI300X d’AMD.
Microsoft is thus equipping itself with a native AI chip for its cloud infrastructure. However, the performance of these chips is not yet known. Indeed, the presentation of the new components is not accompanied by benchmarks. As a result, they cannot be evaluated against third-party solutions available on the market.
Nevertheless, the American IT giant promises significant improvements to its cloud services with these forthcoming additions.