Microsoft has formed a team to develop smaller AI models. In doing so, it wants to make AI a lot cheaper to run than it is today. At the same time, the company could become a lot less dependent on OpenAI.
According to The Information, the new team will be led by Misha Bilenko, CVP Azure AI Cognitive Services at Microsoft. The group will report to Microsoft CTO Kevin Scott. The envisioned Small Language Models (SLMs) should take a lot less computing power to run than OpenAI’s offerings. Currently, virtually all of Microsoft’s AI services run (at least partially) on OpenAI models, such as GPT-3, GPT-4 and GPT-4-Turbo.
These LLMs eat power and water: every 40 outputs of ChatGPT, for example, is estimated to cost a litre of water. Highly expensive Nvidia H100 GPUs are needed to run such models as fast as possible. Other tech giants like Google, AWS and Meta are fighting with OpenAI and Microsoft to get their hands on hundreds of thousands of these chips. There is definitely an advantage to building lighter models, then, which not only cost much less power but also can be run on less imposing hardware.
SLMs: not new, but promising
The goal of the Microsoft team is to produce SLMs that can still rival OpenAI’s models. Late last year they already presented Phi-2, which was trained on high-quality data and has “only” 2.7 billion parameters. At the time, several tests showed that the relatively small model was competitive with much larger competitors such as Meta’s Llama 2-7B or Google’s offerings.
This development is not entirely out of the blue. However, the company’s dependence on OpenAI does prove to be a pain point for Microsoft. After uproar surrounding the governance of OpenAI, it seemed for a while that CEO Sam Altman would make the move to Redmond, but that has since been reversed. In addition, Microsoft had to make do with a supporting role on the board, in which it has no voting rights despite an investment worth billions of dollars.
AI offerings not without problems
Lawsuits against OpenAI and Microsoft may likewise have spurred the latter to operate more independently. With its own models, trained on licensed or free data, Microsoft could ensure that its mountain of Copilots and other AI services can continue to operate if OpenAI’s GPT models are banned by the courts or if any other constraints are imposed. While such a decision is not likely anytime soon, Microsoft cannot simply wait for it to happen.
Also read: Microsoft Copilot Pro misfires at launch with slow performance