“Some people have a way with words, and other people, uh, not have way.” – Steve Martin
The scariest part of moving to a new country is learning a new language. You step off the plane and into a new land with illegible signs, strange customs, and unfamiliar culture. Which way do I go? Whom do I ask for help? Where do I even start?
The strange world of AI/ML is terra incognita for most of us. We’ve heard rumors of its vast richness and unlimited opportunity. It certainly sounds exciting, for those willing to take the plunge. But how do you navigate its unfamiliar territory without a local guide or even a phrasebook to get you started?
Fortunately, the ML-on-IoT landscape is populated by friendly natives willing to lend a hand. They want you to succeed, and they’re willing to donate their time to make the transition as easy as possible. Maybe, one day, you’ll become a productive member of their society. Which means you might also become a paying customer.
Say hello (or Grüß Gott, ¡Hola!, Χαίρετε, xin chào, or こんにちは) to TensAI Flow and Glow, two new programming tools from Eta Compute and NXP, respectively. Although they’re as different as Swedish and Swahili, both exist for the same purpose: to get you conversant with machine-learning coding as painlessly as possible.
Eta Compute, you’ll recall, is the company that produces the ECM3532 microcontroller chip, a two-headed MCU-with-DSP created especially for low-power inference at the edge. The company is convinced that uploading sensor data to a cloud server is the wrong way to do things and has dedicated its hardware and software efforts to overturning that usage model. Chips should be smart enough to collect and analyze sensor data without outside help and without draining your batteries or bank account.
Making the hardware was tough enough, but making it easy to program may have been even tougher. TensAI Flow is the company’s new software toolchain to ease the burden. It takes high-level models created or collected by TensorFlow Lite, Edge Impulse, or ONNX, links them with its neural network zoo, and converts them to ECM35xx binaries. The software bundle also comes with libraries for common functions like image classification, motion detection, and sound processing, as well as drivers for popular sensors. TensAI Flow handles the nontrivial task of dividing up the workload between the chip’s ARM processor and its DSP companion. Finally, it produces runtime executables for FreeRTOS, the lightweight scheduler.
TensAI Flow can’t train your neural network for you, but it significantly eases the task of getting that model onto real hardware. That’s a boon for pioneers in this largely unexplored world of ML-on-IoT programming. It provides a quick reality check for new projects, and it’s a quick-turnaround tool for iterating once you get further along. And it’s free. Some parts of the TensAI Flow package are open-sourced, but most is Eta Compute’s original work, freely licensed as long as it’s for Eta Compute hardware.
Over on the other side of the world, chip behemoth NXP is also polishing up its ML credentials with the release of its Glow compiler for microcontrollers. Glow is a machine-learning compiler originally created at Facebook but now open-sourced and gaining industry support. Like any compiler, it’s language-specific but hardware-agnostic. A generic out-of-the-box Glow compiler will produce acceptable results (like gcc) but nothing to write home about.
That’s where NXP comes in. The company has optimized its Glow compiler for its own line of i.MX RT microcontrollers, which – just like those at Eta Compute – are based on an ARM Cortex-M paired with a DSP core. (Weirdly, NXP doesn’t use its own CoolFlux DSP, opting instead for Tensilica’s HiFi 4 design.) The compiler accepts PyTorch, ONNX, or TensorFlow Lite input, should you decide to go that route. NXP’s Glow implementation includes CMSIS-NN libraries and other chip-specific tweaks to get the best out of your models.
It certainly seems to work. In NXP’s benchmarks, the company showed a 2× improvement in performance just by linking in the CMSIS-NN library, and a huge 13× improvement when the compiler targets the on-chip DSP. Let’s hear it for hardware/software optimization.
As NXP’s Director of Enabling Technologies, Markus Levy, points out, ML code is just middleware — it’s never the entire application, so performance gains mean more headroom for your “real” code. Your ML algorithms may hold your most valuable and hard-won IP, but they’re not what the customer sees. A big jump in compiler efficiency is like getting free CPU cycles to spend elsewhere.
NXP and Eta Compute are both betting big on high-volume ML hardware, and both know that accessible software is the way to move chips. Maybe their new tools can bridge the language barrier and get more developers talking.
