This market research report was originally published at Tractica's website. It is reprinted here with the permission of Tractica.
Qualcomm is swimming upstream and intends to take its 30+ years of experience in power efficient mobile silicon right into the data center with a new purpose-built chip for AI inference acceleration.
The infancy of AI technology is driving several fundamental shifts in how AI processing is accomplished. CPUs gave way to GPUs for their ability to perform millions of tasks in parallel, and inference processing is shifting from the cloud to the edge for applications that require real-time results. Another inevitable shift underway is the move to purpose-built silicon for AI. Qualcomm is positioning itself to take full advantage of these shifts with the announcement of its Cloud AI 100 chip.
Extending Its Expertise to a New Territory
Cloud AI 100 is a dedicated AI inference accelerator that Qualcomm will bring to the hyperscalers and big cloud providers and will sit in a rack in the data center. Clearly, this is a new territory and complete greenfield for a company that has been known for powering the client side of the experience.
But Qualcomm’s expertise in building silicon for clients might be what ultimately gives it the advantage over the 30+ competitors also pursuing dedicated AI accelerators. Power efficiency, process node leadership, and signal processing expertise, along with a scale that few can match, are pieces of a mobile heritage that cloud providers will value in Qualcomm.
Power efficiency is a discipline Qualcomm parlayed from handsets to other clients, including XR devices, laptops, and wearables, and could prove to be very important to cloud providers needing to hit a certain performance threshold at a given power level. Cloud players will want to know how much AI inference performance can be achieved on a rack powered at a certain wattage. What can be achieved on a 20 watt rack? How much inference can a 75 watt rack deliver? The more power efficient the accelerator, the better the answer will be.
The new chip is currently being tested by several hyperscalers and cloud providers, and its actual performance metrics will not be made public until testing is completed. Initial claims pin the performance per watt of the Cloud AI 100 chip at greater than 10x over any other AI accelerator available today. This is still just an unsubstantiated claim until real performance metrics are released. But it’s conceivable given how much easier it is for Qualcomm to scale up from its power efficient starting point than it is for a competitor that has not operated under the same power sipping discipline to scale down to a similar level of power efficiency.
Contributing to its power efficiency story is Qualcomm’s process node leadership, where it has pushed its foundry partners to stay on the technology’s leading edge. Qualcomm even has a history of switching foundries if its partner is unable to maintain pace. That was the case when Qualcomm switched to TSMC in 2018 when its partner at the time, Samsung, was unable to deliver a commercially feasible 7 nm solution for its Snapdragon 855 flagship processor. The nanometer measurement is a key metric in semiconductor fabrication and is an indicator of the space between the transistors on a chip. The smaller the space, the better its performance and power consumption.
Qualcomm’s significant expertise extends to signal processing, which also needs to be accomplished in a power efficient manner for communications. Such efficiency is needed in other areas, as well – such as the camera, video, audio, gestures, and optimizations for XR, where Qualcomm has developed discrete signal processors. Low power signal processing is a competency that can benefit the cloud in AI processing.
What’s more, Qualcomm has scale advantage that few could match. It’s already shipping more than 6 billion chips per year, 700 million of which are Snapdragon systems-on-chip (SoCs), and it has more than 30 foundry sites to further scale behind.
Delivering a Complete Solution
But having the chip is not enough; Qualcomm needs the software stack to go along with it and will need to deliver the tools, the runtimes, and support for the key frameworks. And since each hyperscaler has its own unique software stack, such as Facebook with Glow or Microsoft with ONNX, it’s doubtful all would be supported at the outset. While Qualcomm can deliver the silicon at a nearly unmatched scale, its software will scale on a different timetable.
Given the rate of power consumption growth in today’s data centers is unsustainable, it’s extremely likely that the AI Cloud 100 is just the first of many chips that Qualcomm will develop for data centers.
Kevin Burden
Director of Primary Research, Tractica
