It’s a funny old life when you come to think about it. You never know when something is going to spring up to surprise and delight you or to bite you on the a…sk no questions. As a “surprise and delight” example, I recently received a request to give the keynote presentation at the FPGA Forum in Trondheim, Norway, 9-10 February 2022.
With regard to the “ask no questions” front, I foolishly asked my wife (Gina the Gorgeous) if she would care to accompany me to the aforementioned forum. As a result, she is now preparing for the event as though we were mounting an arctic expedition. I didn’t realize she would require so much equipment in the form of coats, boots, scarves (“I can’t wear the same one every day because people would notice”), hats, and goodness knows what else. I have absolutely no idea where we are going to keep the sled dogs.
I find it hard to believe that this will mark 10 years since I last spoke at the FPGA Forum. As I mentioned in a previous column, every time I blink, another day goes by; every time I sneeze, I lose another week; and every time I… but we digress.
I still vividly remember my 2012 foray to the forum. In addition to being my first visit to Norway, which was jolly exciting in its own right, I was invited to give a guest lecture at the University of Oslo. As part of that side-excursion, I got to visit the Kon-Tiki Museum, which — amongst other things — contains the eponymous raft used by the Norwegian adventurer, explorer, and ethnographer Thor Heyerdahl in his famous Kon-Tiki expedition. Also, I was taken to see Frogner Park with its Vigeland Park installation boasting hundreds of sculptures as well as larger structures such as bridges and fountains. Strolling round the snow- and ice-covered park at midnight in winter was a magical experience I will never forget.
When I come to think about it, I’ve been amazingly lucky in my life. In addition to my outrageous good looks, my legendary sense of fashion, and my sartorial sense of style, I’ve also been fortunate enough to see a lot of the world with other companies picking up the bill (“It’s really the only way to travel, my dear!”). In 2010, for example, two years prior to my Norwegian escapade, I was invited to speak at the Embedded Systems Conference (ESC) in Bangalore, India.
One day in Bangalore, we got to eat at an open-air restaurant in the middle of the city. Each person was presented with a medium-sized metal tray bearing a bunch of smaller metal dishes arranged in an arc around the circumference of the salver, while the central area was reserved for breads and similar solids. In addition to waiting and waiting, waiters were wandering around, each carrying containers of food. Each waiter was in charge of a certain dish, and their mission in life was to keep their dish on your tray fully charged until you turned it over, thereby indicating you could eat no more. The food was delicious! I’ve never tasted the like until… but I’ll get to that in a moment.
One of the things we tend to take for granted these days is our ability to meet people and make friends around the globe by means of the internet. For example, as part of my columns on EETimes.com, Embedded.com, and EEWeb.com, I got to meet Sree Harsha Angara, who was an application engineer (AE) with Cypress Semiconductor. Sree was based in India at that time, which was also around the time I was there, but he didn’t know I was visiting India and I didn’t even know he lived there. Talk about ships that pass in the night.
The way we originally got to connect was that I would often post a “wouldn’t it be cool to do this” column on a Friday afternoon to give people something to think about over the weekend. When I returned to my office the following Monday morning, I would be greeted by an email from Sree showing his solution to my problem, which was invariably based on one of Cypress’s PSoC (programmable system on a chip) devices. (PSoCs include a CPU core along with mixed-signal arrays of configurable integrated analog and digital fabric and peripherals.)
As one example, sometime around 2017, I was toying with the thought of building a brain. Well, a sort of three-dimensional network of simple analog and/or digital artificial neurons, all to be connected using a flywire technique, and all to be mounted in a tall domed glass cylinder. My idea was that the inputs to this network would be fed by various sensors, and it would respond to light, sound, and changes in the environment. As part of this, I suggested that it would be nice to have a ring of LEDs around the bottom, accompanied by six microphones, and to use the LEDs to reflect the source of any sounds (similar to the top of an Amazon Echo or Amazon Dot). Well, blow me down if Sree didn’t rise to the challenge and create a solution that very weekend (I think this one earned him the “Cypress Maker of the Month” award).
A ring of 60 tricolor LEDs with six microphones, all powered by a PSoC (Image source: Sree Harsha Angara)
By this time, Cypress had moved Sree to Silicon Valley in the role of Staff AE, which is like an AE on steroids. Happily, Sree and I eventually got together when I was presenting at ESC Silicon Valley in 2019. It’s a strange thing to finally meet and greet someone you’ve known for years but you’ve never actually met and you don’t even know what they look like. And, just when I thought things couldn’t get any better, I mentioned to Sree about the awesome meal I’d enjoyed in Bangalore, and he replied that there was a similar restaurant just a few miles away from the conference, so we hopped in his car and headed there at full speed. O-M-G! The food was just as good as I remembered it to be.
My friend Sree (Image source: Max Maxfield)
Of course, a lot of Poohsticks have passed under the bridge since those halcyon days. As one obvious example, even though our meeting was only around two years ago as I pen these words, we’ve had a worldwide pandemic since then. Also, slap dab in the middle of that pandemic, in April 2020, Infineon closed its acquisition of Cypress Semiconductor, where Sree is now a Product Marketing Manager.
I’ve mentioned before how my dear old mother will start a story along the lines of, “I met old Mrs Pennywhistle at the fishmongers the other day. You remember her — she was the one with three daughters — the eldest girl married an Australian taxidermist and they have two sons, both of whom are allergic to bananas, and…” As the hours go by, you see the fear in her audience’s eyes that the story will never end, and then suddenly and triumphantly she brings it all back home with something like, “So that’s why you should never pickle a pilchard!” Well, I’m about to do something similar…
A couple of weeks ago I received an email from the PR folks at Infineon saying, “Hi Max, the combination of artificial intelligence (AI) and the Internet of Things (IoT) — known as the Artificial Intelligence of Things (AIoT) — provides machine learning capabilities in connected devices, enabling them to perform intelligent tasks. According to Markets and Markets, the AIoT market is expected to increase from $5.1 billion USD in 2019 to $16.2 billion USD by 2024, growing at a CAGR of 26%. The next logical question is how do we protect these intelligent endpoints and the increasingly valuable data that they will have access to? On May 19, Infineon will announce their solution that can provide an answer. Happy to share additional details and schedule a briefing for you with an Infineon spokesperson.”
I of course said that I would love to hear more, so they set up a video conference call with the Infineon spokesperson, who turned out to be… you got it… none other than my old chum Sree!
Eeek! I fear I’ve been waffling too much, so I’ll try to speed up a little. Let’s start with the ModusToolbox, which is a collection of easy-to-use software and tools enabling rapid development of Infineon PSoC MCUs, covering applications from embedded sense and control to wireless and cloud-connected systems.
Next, let’s note that we are currently seeing a rapid increase in connected devices along with an exponential growth in the amount of data they generate. More data is good (it provides better insights and performance) and bad (in addition to security and privacy issues, there are bandwidth and latency concerns). One way to alleviate these issues and concerns is to move machine learning (ML) workloads to the edge.
In order to address this, the folks at Infineon have introduced ModusToolbox ML, which enables deep learning-based workloads on Infineon’s PSoC MCUs. Sree walked me through a typical embedded ML workflow. As we see in the image below, ML at the edge is focused on inference.
ML at the edge is focused on inference (Image source: Infineon)
ModusToolbox ML is a new feature in the ModusToolbox that provides middleware, software libraries, and special tools for designers to evaluate and deploy deep learning-based ML models. This feature allows seamless integration with the existing frameworks available in ModusToolbox so that ML workloads can be easily integrated into secured AIoT systems. ModusToolbox ML allows developers to use their preferred deep learning framework, such as TensorFlow, to be deployed directly to PSoC MCUs. In addition, the feature helps designers optimize the model for embedded platforms to reduce size and complexity, as well as to validate performance against test data.
Typical embedded ML workflow (Image source: Infineon)
So, there we have it. In a nutshell, as the IoT scales, massive amounts of data are being generated at the edge. TinyML involves shrinking deep learning networks to fit on tiny hardware resources such as microcontrollers. Enabled by TinyML, AIoT is a natural evolution, where acting on data locally helps manage data privacy, latency, and overall system reliability. ModusToolbox ML allows developers to take their trained networks and optimize them and deploy them in embedded systems. Tra-la!
However, I have a nagging thought that’s niggling at the back of (what I laughingly call) my mind. Remembering how prolific Sree used to be using the original ModusToolbox to create non-AI/ML-based solutions to the problems I posed, I wonder what he’s going to come up with now he has the power of ModusToolbox ML at his disposal. I think we should all be very afraid. How about you? What are your thoughts about all of this?
One thought on “Modus Toolbox ML, TinyML, and the AIoT”