Developers will be able to integrate Microsoft’s AI applications into their own apps through the Copilot Runtime, which contains a library with many AI APIs.
It is by no means a secret that Microsoft has sky-high expectations for the AI applications it is making available for Windows. The company is eager to see developers take full advantage of all the new and existing AI applications under the hood of Windows 11 and has announced that its Copilot Runtime allows developers to use this AI for their own applications.
This runtime primarily ensures smooth operation for the more than 40 AI models that Windows runs in the background. However, it also provides opportunities for developers eager to use Microsoft’s AI applications in their own software. In the company’s own words, this makes Windows the ‘most open platform for AI’.
Developers can use Windows AI functionality in their own apps
The Windows Copilot Runtime contains an API library that developers can integrate when developing their own apps. This gives them access to Windows AI applications and the associated frameworks and toolchains, i.e. the set of utilities, debuggers, compilers and other components needed for software development.
All of this is available on-device, allowing developers to create applications using local AI. That way, they don’t have to rely on Internet access or remote servers to process data. This approach can lead to faster response times, improved privacy and overall user experience, but it also requires capable hardware.
Tip: These major computer brands have already announced their own Copilot+ pc’s
According to Pavan Davuluri, chief of Microsoft’s Windows & Surface division, developers will be able to use Studio Effects, Live Captions Translations, OCR (digitizing documents), Recall with User Activity, and Phi Silica, a small-language model of 3.3 billion parameters, among other things, via the API library in Copilot Runtime. The Verge reports that all this will be available in June.
Studio Effects integrated into Whatsapp
For example, Meta already integrates Windows Studio Effects into WhatsApp. It thus enables background blurring and simulates constant eye contact in video calls (so it looks like a speaker is looking at the viewer when they might actually have their eyes focused on notes).
Microsoft introduced the Recall feature prior to the Build conference. This feature allows Copilot+ PCs to document and recall user activity through a semantic index that understands the relationship between documents. Developers can use this technology to build similar features into their own applications.
According to Microsoft, this should allow app users to pick up where they left off more quickly. “This integration helps users pick up where they left off in your app, improving app engagement and users’ seamless flow between Windows and your app”, claims Davuluri.
Also read: Microsoft announces Copilot+ AI PCs and Recall GenAI features