Qualcomm is introducing a new line of chips today that are custom-made for smart cameras. The new line is called the Vision Intelligence Platform, and it’s meant to get AI features, extreme low-light image processing, image stabilization, and computer vision abilities into more cameras — and to do it better than a repurposed smartphone chip could.
The chips are meant for pretty much any camera that can make use of intensive processing features: sports cameras, security cameras, 360-degree cameras, and even robots (say, something like a Roomba) that need to navigate, among others. The chips support 4K video, multiple video streams at lower resolutions, 360-video stitching, WQHD displays, obstacle avoidance, and the creation of video highlight reels for action cameras.
It’ll be up to hardware manufacturers who use these chips to take advantage of those features and build them out. But Qualcomm’s goal is to make it easier for hardware companies to implement them, and to allow them to be implemented better than they could with other chips. Because there’s dedicated hardware for a lot of these features, Qualcomm says they’ll use up less power while performing better than they would if running on a more general-purpose chip.
The first two chips in the line, the QCS605 and QCS603 (no relation to the Snapdragon 600 series), are designed for Wi-Fi cameras, since neither has an LTE radio. The 605 supports 4K video at 60 fps, and the 603 supports 4K video at 30 fps.
This isn’t Qualcomm’s first attempt at offering chips for smart cameras. It’s done this in the past, but it’s always used repurposed Snapdragon chips before — one ended up in the Nest Cam IQ, for instance. This is the first time Qualcomm has offered chips specifically built for smart cameras, though, and that means future generations of these products could start to advance a bit faster.
Qualcomm has already begun sampling out the 605 chips to hardware partners and expects products using them to come out in the second half of the year.
QCOM Vision Intelligence Platform – QCS605 SoC
Qualcomm’s chips already power virtually every Android smartphone on the market, but the company has long set its sights beyond this core market and is increasingly looking at IoT as one of its next major growth areas. Earlier this year it launched its new embedded platform for IoT developers; today, it’s introducing two new systems-on-a-chip for IoT, the QCS605 and QCS603, that combine a multicore ARM processor with the company’s AI engine and an image signal processor.
The standard use for these chips is in smart security cameras for both consumers and industry, as well as sports cameras, wearable cameras, VR, robotics and smart displays — anywhere you need a good amount of computing power at the edge, as well as the ability to interpret images and run pre-trained machine learning models.
Qualcomm’s IoT journey started a few years ago. Early on, this was mostly about re-purposing the company’s Snapdragon mobile chips for this market. As Qualcomm’s VP of product management Seshu Madhavapeddy told me, the company now believes that it has reached a critical mass of business in IoT — and thanks to that, it now also has a far better understanding of the market’s needs. Instead of just reusing and tweaking existing systems, Qualcomm is now looking to create purpose-built chipsets that still leverage the building blocks of its smartphone tech but also feature custom designs.
For these new chips, that custom design focuses mostly on the AI engine. That part of the chip can handle 2.1 trillion operations per second for neural network inferencing. That’s only a little bit slower than the promised performance of a Mobileeye EyeQ4 chip. As Madhavapeddy stressed, it’s far more efficient to bring inference to the edge, both in terms of latency and bandwidth. There’s no need for the data to make a roundtrip to the data centre, after all. “We anticipated that trend, we have seen that trend, and we are catering to that,” he said.
Developers and data scientists can create their models using any of the standard machine learning frameworks, and Qualcomm will offer an SDK to port those over to its chips.
All that AI power is only useful when you have good data to work with, so the team also focused strongly on the actual image processing part of the chips. “The camera tech uses the best of the camera tech we have at Qualcomm,” Madhavapeddy said. But while the underlying tech is similar, the kind of use cases the team anticipates don’t quite match the expectations of smartphone users. You want to have great low-light video performance in both IoT and consumer cameras, for example, but in an IoT device like a security camera, your focus isn’t on aesthetics. Similarly, the image stabilization algorithms of a sports camera that you’ll attach to your mountain bike need to be tuned differently from those of a smartphone.
Once you have all those building blocks in place, you also need to offer developers the ability to run their software on them, of course. “Regardless of what product you are building, our customers add a lot of different software to differentiate the product they are building,” said Madhavapeddy. For a VR camera, that’s the software that combines the images from all the cameras, for example.
To help developers and OEMs build on top of these new SoCs, Qualcomm also today announced a new VR camera reference design in cooperation with Altek based on the QCS605, as well as an industrial security camera reference design based on the QCS603.
Both systems are sampling now, and we’ll likely see the first products based on them in the second quarter of this year.
