Microsoft this week unveiled a new deep learning acceleration platform.
Codenamed Project Brainwave, the company says it “achieves a major leap forward in both performance and flexibility for cloud-based serving of deep learning models.”
“We designed the system for real-time AI, which means the system processes requests as fast as it receives them, with ultra-low latency,” explains Doug Burger, a distinguished engineer at Microsoft. “Real-time AI is becoming increasingly important as cloud infrastructures process live data streams, whether they be search queries, videos, sensor streams, or interactions with users.”
Project Brainwave takes a different approach, providing a design that scales across a range of data types, with the desired data type being a synthesis-time decision. We architected this system to show high actual performance across a wide range of complex models, with batch-free execution. Our system, designed for real-time AI, can handle complex, memory-intensive models such as LSTMs, without using batching to juice throughput.
Burger says Microsoft is working to bring this real-time AI system to users in Azure.
“In the near future, we’ll detail when our Azure customers will be able to run their most complex deep learning models at record-setting performance,” he says. “With the Project Brainwave system incorporated at scale and available to our customers, Microsoft Azure will have industry-leading capabilities for real-time AI.”