Video surveillance, CCTV, camera-toting drones, cellphone video, stoplight cameras – they’re everywhere! It seems as though no public space isn’t being recorded, filed, uploaded, and possibly analyzed for malfeasance. The common factor in all these scenarios is digital cameras.

And what do all digital cameras need? Lots of storage, lots of bandwidth, and lots of processing power. Grabbing frame after frame of unrefined, uncompressed video isn’t interesting. You need to massage the video before it’s useful. That means some combination of white balance, edge detection, smoothing, compressing, artefact reduction, and possibly image recognition. That’s a lot of work on a lot of pixels, in very little time.

It’s precisely that workload that has kept a portion of the microprocessor industry in business. Crunching hours of video in minutes of real time is not the work of a lightweight processor. In fact, it’s one of the toughest tasks you can toss at a standard RISC or CISC CPU. Just try running your security cameras on an old PC and see how quickly it bogs down. Spreadsheets and word processors are one thing; spotting intruders at 2:00 AM is quite another.

This sounds like a business opportunity! And indeed, it is. There are any number of ways to handle video processing, from DSPs and FPGAs to specialized processors. And a leading light in the latter category is Movidius. 

We first saw Movidius in these pages about four years ago (“The Moving Picture Show”) when the company introduced Myriad 1, its first take at a custom video-processing processor. Since that time, the company has grown up a whole lot and is now ready to unleash Myriad 2, a vastly improved version.

Movidius calls its Myriad 2 chip a “computational imaging and visual awareness” processor, a multipart description that just rolls off the tongue, doesn’t it? If we parse the phraseology, we can glean the “computational” part because it’s a processor. That means you program it, rather than just wiring it up to your CMOS image sensors. It’s a multi-core chip, with several SHAVE processor cores working in collaboration to crack difficult video-processing tasks. (And before you ask, SHAVE stands for “streaming hybrid architecture vector engine” because it is – ahem – on the bleeding edge of video processing. Aren’t you glad you didn’t ask?)

The “visual awareness” part is where the usage case gets a little weird. Myriad 2 is so good at digesting real-time video that it can capture a full 360 degrees all at once and make sense of it. That allows you to do interesting things like create a sort of wraparound GoPro that sees in all directions simultaneously. In fact, one company has done just that, with the video evidence here.

But what if your needs extend beyond posting extreme sports videos? Myriad’s real-time image processing allows you to do somewhat more practical things, like scan a room’s interior dimensions simply by waving a camera around. Shooting video of a room is trivially simple – any smartphone can do that – but making sense of that video is another matter, and that’s where Myriad 2 comes in. By preprocessing the video, the Movidius chip can detect parallax and, thus, distance. Wave the camera around enough to get a good view of everything from all angles, and Myriad 2 can help you put together a pretty accurate 3D model of the entire visible space. Here again, one company is showing how Myriad could do just that. 

Finally, there are the non-video applications for video. Capturing images doesn’t always have to lead to another YouTube upload. Instead, Myriad 2 can be part of a cool, gee-whiz user interface based on gestures and virtual objects. Want to go for the full Minority Report experience, waving your hands in front of a computer screen? You’re going to need some video-capture hardware for that (or special sensor-laden gloves), and that’s where Myriad 2 shines. The example here shows how medical professionals might interact with virtual organs without getting their hands dirty.

Science-fiction author Arthur C. Clarke famously said, “any sufficiently advanced technology is indistinguishable from magic,” and I can’t help thinking that future doctors may soon be waving their hands in the air over their patients to effect surgery. If that doesn’t conjure up images of voodoo and magic, I don’t know what will.

Movidius is a little shy about the technical details of Myriad 2, apart from confirming that it sports an even dozen of its 128-bit SHAVE CPU cores, two additional 32-bit RISC processors, and a complex shared-memory fabric. Indeed, they seem more proud of their memory controller than of the processors; it’s the key to keeping Myriad 2’s power consumption down.

Any video processing involves a lot of pixels and therefore a lot a bitwise comparisons. Keeping the data movement to a minimum is the trick. Typical microprocessors and DSPs move data into and out of on-chip registers to do their comparison or arithmetic work, and those to-and-fro transactions burn up time as well as energy. Movidius focused its attention on minimizing motion (ironically enough), so the SHAVE processor cores in Myriad 2 use a unique (and mostly secret) method of sharing data without a lot of gratuitous back and forth.

As a result, the company says Myriad 2 delivers a 20-fold improvement in “processing efficiency” over its predecessor. Okay… but what exactly is processing efficiency and how do you measure it? Are we talking MIPS/watt, operations/second, furlongs/fortnight, or what? The metrics can be a bit vague, but the company explained that the 20x improvement was in “energy consumed per function,” and was therefore a ratio of work performed over energy consumed. “Think joules, not watts,” they suggested.

That’s fair. Most Myriad 2 applications will be in mobile devices (isn’t everything?), and that means battery power. Which means efficiency is almost as important as all-out performance. Yes, real-time video processing is hard, but if it eats up all my battery before I complete that awesome flying-squirrel-suit jump off of El Capitan, what good is it?

Programming Myriad 2 won’t be easy. Coding multicore processors never is, but Movidius gives you a head start with a bunch of prewritten routines along with its vectorizing compiler.

So whether you’re into surgery or selfies, surveillance or speeding tickets, Myriad 2 will likely play a part in it. Smile for the ubiquitous cameras.