Ever since the debut of Cortana within Windows 10, Microsoft has tried to inject Windows with artificial intelligence. Now it’s getting even more serious with a new API, called Windows ML, that will tap your CPU or GPU to make Windows and its apps even smarter.
Windows ML will debut within the “next major update to Windows 10,” presumably the “Redstone 4” update set to be released to hardware partners this month. Since it’s an API, every developer who writes apps for Windows will be able to take advantage of the new AI capabilities.
In the future, Microsoft said, Windows ML might even take advantage of an entirely new chip: a machine vision or visual processing chip designed by a subsidiary of Intel, called Movidius. Microsoft said that it is working closely with Movidius to support its 2485 VPU Accelerator, which could work in conjunction with or simply replace today’s webcams.
Microsoft
A practical use of Windows ML would be to visually inspect circuit boards for defects, Microsoft said.
What this means for you: On one hand, this seems like a smart, evolutionary step for Microsoft. You can search your Photos app for “beach” pictures, for example, or identify faces. And Cortana offers up suggestions about when to leave for appointments. But allowing developers to inject their own ideas makes even more sense. On the other hand, Microsoft will probably have to placate user groups like gamers, who will worry about Windows robbing their GPU of resources while playing a game.
Evolving Windows to make it smarter
“In Windows we already use AI in a broad set of ways,” said Kam VedBrat, a group program manager with Microsoft, who explained Windows ML during a Microsoft developer day webcast. “We use it for handwriting recognition, isolate security threats, and to power services like Cortana… So how do we think about bringing AI to Windows scale?”
Microsoft
Kam VedBrat, a group program manager with Microsoft, holds up the Movidius chip.
The answer appears to be a combination of local processing and sending data back up into Microsoft’s Azure cloud for processing. VedBrat said that in certain situations, data sets would be too large or too expensive to send back to Microsoft; in others, the device might simply have an active data connection. In that case, he said, the device should be able to supply the AI processing power itself.
“When you use AI in Windows devices, you get immediate results,” VedBrat said. “Windows uses the hardware capabilities on the devices to deliver great performances with low latency.”
Microsoft
VedBrat also supplied this Windows dxdiag screen to show the Movidius chip in action.
Windows ML will have the capability to use both the CPU as well as the GPU, with support for advanced CPU instruction sets like Intel’s AVX512. As a demonstration, VedBrat showed how a CPU could be used to manipulate an static image, giving it an arty, candy-like appearance (seen at the top of this page). But when the app was run on a Surface Book’s internal GPU, that app was able to apply the same filter to live video captured by the Book’s camera, though at low frame rates.
In the future, Microsoft hopes to bring ML capabilities to devices like the Movidius visual processor, a version of which Intel hopes will power tomorrow’s self-driving cars. Intel supplied one of the first RealSense depth cameras for the PC, which evolved into the Windows Hello-branded biometric recognition cameras that have become somewhat common on today’s PCs.