feature article
Subscribe Now

Modus Toolbox ML, TinyML, and the AIoT

It’s a funny old life when you come to think about it. You never know when something is going to spring up to surprise and delight you or to bite you on the a…sk no questions. As a “surprise and delight” example, I recently received a request to give the keynote presentation at the FPGA Forum in Trondheim, Norway, 9-10 February 2022.

With regard to the “ask no questions” front, I foolishly asked my wife (Gina the Gorgeous) if she would care to accompany me to the aforementioned forum. As a result, she is now preparing for the event as though we were mounting an arctic expedition. I didn’t realize she would require so much equipment in the form of coats, boots, scarves (“I can’t wear the same one every day because people would notice”), hats, and goodness knows what else. I have absolutely no idea where we are going to keep the sled dogs.

I find it hard to believe that this will mark 10 years since I last spoke at the FPGA Forum. As I mentioned in a previous column, every time I blink, another day goes by; every time I sneeze, I lose another week; and every time I… but we digress.

I still vividly remember my 2012 foray to the forum. In addition to being my first visit to Norway, which was jolly exciting in its own right, I was invited to give a guest lecture at the University of Oslo. As part of that side-excursion, I got to visit the Kon-Tiki Museum, which — amongst other things — contains the eponymous raft used by the Norwegian adventurer, explorer, and ethnographer Thor Heyerdahl in his famous Kon-Tiki expedition. Also, I was taken to see Frogner Park with its Vigeland Park installation boasting hundreds of sculptures as well as larger structures such as bridges and fountains. Strolling round the snow- and ice-covered park at midnight in winter was a magical experience I will never forget.

When I come to think about it, I’ve been amazingly lucky in my life. In addition to my outrageous good looks, my legendary sense of fashion, and my sartorial sense of style, I’ve also been fortunate enough to see a lot of the world with other companies picking up the bill (“It’s really the only way to travel, my dear!”). In 2010, for example, two years prior to my Norwegian escapade, I was invited to speak at the Embedded Systems Conference (ESC) in Bangalore, India.

One day in Bangalore, we got to eat at an open-air restaurant in the middle of the city. Each person was presented with a medium-sized metal tray bearing a bunch of smaller metal dishes arranged in an arc around the circumference of the salver, while the central area was reserved for breads and similar solids. In addition to waiting and waiting, waiters were wandering around, each carrying containers of food. Each waiter was in charge of a certain dish, and their mission in life was to keep their dish on your tray fully charged until you turned it over, thereby indicating you could eat no more. The food was delicious! I’ve never tasted the like until… but I’ll get to that in a moment.

One of the things we tend to take for granted these days is our ability to meet people and make friends around the globe by means of the internet. For example, as part of my columns on EETimes.com, Embedded.com, and EEWeb.com, I got to meet Sree Harsha Angara, who was an application engineer (AE) with Cypress Semiconductor. Sree was based in India at that time, which was also around the time I was there, but he didn’t know I was visiting India and I didn’t even know he lived there. Talk about ships that pass in the night.

The way we originally got to connect was that I would often post a “wouldn’t it be cool to do this” column on a Friday afternoon to give people something to think about over the weekend. When I returned to my office the following Monday morning, I would be greeted by an email from Sree showing his solution to my problem, which was invariably based on one of Cypress’s PSoC (programmable system on a chip) devices. (PSoCs include a CPU core along with mixed-signal arrays of configurable integrated analog and digital fabric and peripherals.)

As one example, sometime around 2017, I was toying with the thought of building a brain. Well, a sort of three-dimensional network of simple analog and/or digital artificial neurons, all to be connected using a flywire technique, and all to be mounted in a tall domed glass cylinder. My idea was that the inputs to this network would be fed by various sensors, and it would respond to light, sound, and changes in the environment. As part of this, I suggested that it would be nice to have a ring of LEDs around the bottom, accompanied by six microphones, and to use the LEDs to reflect the source of any sounds (similar to the top of an Amazon Echo or Amazon Dot). Well, blow me down if Sree didn’t rise to the challenge and create a solution that very weekend (I think this one earned him the “Cypress Maker of the Month” award).

A ring of 60 tricolor LEDs with six microphones, all powered by a PSoC (Image source: Sree Harsha Angara)

By this time, Cypress had moved Sree to Silicon Valley in the role of Staff AE, which is like an AE on steroids. Happily, Sree and I eventually got together when I was presenting at ESC Silicon Valley in 2019. It’s a strange thing to finally meet and greet someone you’ve known for years but you’ve never actually met and you don’t even know what they look like. And, just when I thought things couldn’t get any better, I mentioned to Sree about the awesome meal I’d enjoyed in Bangalore, and he replied that there was a similar restaurant just a few miles away from the conference, so we hopped in his car and headed there at full speed. O-M-G! The food was just as good as I remembered it to be.

My friend Sree (Image source: Max Maxfield)

Of course, a lot of Poohsticks have passed under the bridge since those halcyon days. As one obvious example, even though our meeting was only around two years ago as I pen these words, we’ve had a worldwide pandemic since then. Also, slap dab in the middle of that pandemic, in April 2020, Infineon closed its acquisition of Cypress Semiconductor, where Sree is now a Product Marketing Manager.

I’ve mentioned before how my dear old mother will start a story along the lines of, “I met old Mrs Pennywhistle at the fishmongers the other day. You remember her — she was the one with three daughters — the eldest girl married an Australian taxidermist and they have two sons, both of whom are allergic to bananas, and…” As the hours go by, you see the fear in her audience’s eyes that the story will never end, and then suddenly and triumphantly she brings it all back home with something like, “So that’s why you should never pickle a pilchard!” Well, I’m about to do something similar…

A couple of weeks ago I received an email from the PR folks at Infineon saying, “Hi Max, the combination of artificial intelligence (AI) and the Internet of Things (IoT) — known as the Artificial Intelligence of Things (AIoT) — provides machine learning capabilities in connected devices, enabling them to perform intelligent tasks. According to Markets and Markets, the AIoT market is expected to increase from $5.1 billion USD in 2019 to $16.2 billion USD by 2024, growing at a CAGR of 26%. The next logical question is how do we protect these intelligent endpoints and the increasingly valuable data that they will have access to? On May 19, Infineon will announce their solution that can provide an answer. Happy to share additional details and schedule a briefing for you with an Infineon spokesperson.”

I of course said that I would love to hear more, so they set up a video conference call with the Infineon spokesperson, who turned out to be… you got it… none other than my old chum Sree!

Eeek! I fear I’ve been waffling too much, so I’ll try to speed up a little. Let’s start with the ModusToolbox, which is a collection of easy-to-use software and tools enabling rapid development of Infineon PSoC MCUs, covering applications from embedded sense and control to wireless and cloud-connected systems.

Next, let’s note that we are currently seeing a rapid increase in connected devices along with an exponential growth in the amount of data they generate. More data is good (it provides better insights and performance) and bad (in addition to security and privacy issues, there are bandwidth and latency concerns). One way to alleviate these issues and concerns is to move machine learning (ML) workloads to the edge.

In order to address this, the folks at Infineon have introduced ModusToolbox ML, which enables deep learning-based workloads on Infineon’s PSoC MCUs. Sree walked me through a typical embedded ML workflow. As we see in the image below, ML at the edge is focused on inference.

ML at the edge is focused on inference (Image source: Infineon)

ModusToolbox ML is a new feature in the ModusToolbox that provides middleware, software libraries, and special tools for designers to evaluate and deploy deep learning-based ML models. This feature allows seamless integration with the existing frameworks available in ModusToolbox so that ML workloads can be easily integrated into secured AIoT systems. ModusToolbox ML allows developers to use their preferred deep learning framework, such as TensorFlow, to be deployed directly to PSoC MCUs. In addition, the feature helps designers optimize the model for embedded platforms to reduce size and complexity, as well as to validate performance against test data.

Typical embedded ML workflow (Image source: Infineon)

So, there we have it. In a nutshell, as the IoT scales, massive amounts of data are being generated at the edge. TinyML involves shrinking deep learning networks to fit on tiny hardware resources such as microcontrollers. Enabled by TinyML, AIoT is a natural evolution, where acting on data locally helps manage data privacy, latency, and overall system reliability. ModusToolbox ML allows developers to take their trained networks and optimize them and deploy them in embedded systems. Tra-la!

However, I have a nagging thought that’s niggling at the back of (what I laughingly call) my mind. Remembering how prolific Sree used to be using the original ModusToolbox to create non-AI/ML-based solutions to the problems I posed, I wonder what he’s going to come up with now he has the power of ModusToolbox ML at his disposal. I think we should all be very afraid. How about you? What are your thoughts about all of this?

One thought on “Modus Toolbox ML, TinyML, and the AIoT”

Leave a Reply

featured blogs
Dec 2, 2024
The Wi-SUN Smart City Living Lab Challenge names the winners with Farmer's Voice, a voice command app for agriculture use, taking first place. Read the blog....
Dec 3, 2024
I've just seen something that is totally droolworthy, which may explain why I'm currently drooling all over my keyboard....

Libby's Lab

Libby's Lab - Scopes Out Silicon Labs EFRxG22 Development Tools

Sponsored by Mouser Electronics and Silicon Labs

Join Libby in this episode of “Libby’s Lab” as she explores the Silicon Labs EFR32xG22 Development Tools, available at Mouser.com! These versatile tools are perfect for engineers developing wireless applications with Bluetooth®, Zigbee®, or proprietary protocols. Designed for energy efficiency and ease of use, the starter kit simplifies development for IoT, smart home, and industrial devices. From low-power IoT projects to fitness trackers and medical devices, these tools offer multi-protocol support, reliable performance, and hassle-free setup. Watch as Libby and Demo dive into how these tools can bring wireless projects to life. Keep your circuits charged and your ideas sparking!

Click here for more information about Silicon Labs xG22 Development Tools

featured chalk talk

Accelerating Tapeouts with Synopsys Cloud and AI
Sponsored by Synopsys
In this episode of Chalk Talk, Amelia Dalton and Vikram Bhatia from Synopsys explore how you can accelerate your next tapeout with Synopsys Cloud and AI. They also discuss new enhancements and customer use cases that leverage AI with hybrid cloud deployment scenarios, and how this platform can help CAD managers and engineers reduce licensing overheads and seamlessly run complex EDA design flows through Synopsys Cloud.
Jul 8, 2024
43,987 views