Microsoft announces personal AI chip that can compete with Nvidia

Microsoft It unveiled two chips at the Ignite conference in Seattle on Wednesday.

First, its Maia 100 artificial intelligence chip, which it competed against Nvidia A highly sought-after AI graphics processing unit. The second, the Cobalt 100 Arm chip, is aimed at general computing tasks and can compete with Intel processors.

Cash-rich tech companies have begun giving their customers more options for cloud infrastructure they can use to run applications. Ali Baba, Amazon and Google have been doing this for years. Microsoft, with about $144 billion in cash at the end of October, had a 21.5% share of the cloud market in 2022, trailing only Amazon. One assessment.

Virtual machine instances running on Cobalt chips will become commercially available through Microsoft's Azure cloud in 2024, Rani Borkar, corporate vice president, told CNBC in an interview. He did not specify a release date for the Maia 100.

Google In 2016, it announced its original tensor processing unit for AI. Amazon Web Services revealed a Graviton Arm-based chip and Inferentia AI processor in 2018 and announced Trainium for training models in 2020.

Special AI chips from cloud providers can help meet demand during GPU shortages. But Microsoft and its peers in cloud computing aren't going to let companies buy servers that contain their chips, unlike Nvidia or AMD.

The company built its chip for AI computing based on customer feedback, Borkar explained.

Microsoft is testing how Maia 100 fits the needs of its Bing search engine AI chatbot, GitHub Copilot coding assistant and GPT-3.5-Turbo, Microsoft's supported OpenAI large language model, Borkar said. OpenAI fed its language models large amounts of information from the Internet, and they can generate email messages, summarize documents, and answer questions with just a few words of human instruction.

The GPT-3.5-Turbo model works in OpenAI's ChatGPT assistant, which became popular shortly after it became available last year. Companies then quickly moved to add similar chat capabilities to their software, driving demand for GPUs.

“We worked all over the world and [with] All of our different suppliers to help improve our supply position and support our many customers and the demand they've placed on us,” said Colette Kress, Nvidia's chief financial officer, at the Evercore conference in New York in September.

OpenAI has had it before trained models on an Nvidia GPU in Azure.

In addition to the Maia chip design, Microsoft developed custom liquid cooling hardware called Sidekicks that fit into the racks next to the racks containing the Maia servers. The company can install server racks and Sidekick racks without the need for remodeling, the spokesperson said.

With GPUs, making the most of limited data center space can present challenges. Companies sometimes place multiple servers that contain GPUs at the bottom of a rack, like “orphans,” to avoid stacking the racks from top to bottom, said Steve Tuck, founder and CEO of server Oxide Computer. Companies sometimes add cooling systems to lower temperatures, Tuck said.

Microsoft may see faster adoption of Cobalt processors than Gaia AI chips if Amazon's experience is any guide. Microsoft is testing its Teams app and Azure SQL database service on Cobalt. So far, they've performed 40% better than Azure's existing Arm-based chips from startup Ampere, Microsoft said.

Over the past year and a half, as prices and interest rates have risen, many companies have looked for ways to make their cloud spending more efficient, and for AWS customers, Graviton was one of them. All of AWS's top 100 customers now use Arm-based chips, which can deliver a 40% price improvement, said vice president Dave Brown.

Moving from GPUs to AWS Trainium AI chips can be more difficult than moving from Intel Xeons to Gravitons. Each model of artificial intelligence has its own characteristics. A lot of people have worked to get different tools to work on Arm because they're so prevalent in mobile devices and it's less suited to silicon for AI, Brown said. But over time, he said, he expects organizations to see similar pricing with Trainium compared to GPUs.

“We've shared these specifications with the ecosystem and with many of our partners in the ecosystem, which benefits all of our Azure customers,” he said.

Borkar said he didn't have details on the Maya's performance compared to alternatives like Nvidia's H100. On Monday, Nvidia announced that its H200 will begin shipping in the second quarter of 2024.

WATCHING: Nvidia misses 10th straight set of gains, driven by new AI chip announcement