We’ve elaborated at length in these pages on the remarkable capabilities FPGAs bring to the world of embedded vision and vision analytics. With enormous numbers of new applications demanding embedded vision capabilities at the endpoint and edge, there’s a huge green field of opportunity out there, just waiting for FPGAs to take over. The ability of FPGAs to accelerate AI tasks on a miserly power budget, perform various video filtering and processing tasks, and adapt to handle basically any interface standard for the myriad sensors, displays, and other related hardware out there is unmatched by any other technology on the market.

But, adoption of FPGAs continues to be limited by the difficulty – both perceived and real – of customizing the devices for each particular application. Historically, FPGA use has been the exclusive purview of engineering teams with expertise in HDL design and FPGA experience. The skill and knowledge required to create Verilog descriptions of custom hardware, simulate and synthesize that code, and complete place and route as well as timing closure – has been a major barrier to entry for the FPGA party for the last three decades. 

FPGA companies are acutely aware of this, of course, and perhaps the most effective weapon they have to combat the problem is the pre-engineered “stack” for specific classes of applications. A stack is an almost-complete sample application – including development boards, reference designs, relevant IP blocks, software, firmware, and whatever else you need to get to your application’s version of “Hello World.” With a stack, you can usually be up and running with something that approximates your project in a matter of hours or even minutes from first powering up your development kit. Then, it’s just a matter of modifying the already-working system to include your own special sauce. While this doesn’t do away with the need for hardware and FPGA expertise altogether, it certainly can dramatically reduce the learning curve.

Lattice Semiconductor has just announced mVision, a new “stack” aimed at machine vision in power-constrained edge and endpoint designs. Lattice has long led the field in ultra-low power FPGAs (think fractions of watts). They have shipped over a billion devices into systems such as smartphones, cameras, and other battery-powered and portable devices. Lattice estimates the machine vision market will grow to over $18B by 2025, and they figure mVision will help their aim to claim a chunk of that loot in the high-volume, low-power end of the spectrum.

mVision supports three different families of Lattice FPGAs – their MIPI-bridging Crosslink and Crosslink-plus families (with up to 6K LUTs, 150 mW power dissipation, and 2.5×2.5 mm packaging), the recently-announced Lattice Nexus Crosslink-NX (built on 28nm FDSOI technology, with 40K LCs, 0.3 W power dissipation, and 6×6 mm packaging) and the larger and more powerful ECP5 family (with 85K LUTs, 1 W power dissipation, and 10×10 mm packaging). This gives us a nice range of capability, power dissipation, and form factor depending on our project’s constraints – all under the same mVision umbrella.

Lattice’s video interface platform (VIP) development boards are clever mix-and-match stacked modules that let you pair your choice of FPGA, camera interface, and IO interfaces. Supported interfaces include MIPI, and LVDS image sensors, MIPI, eDP, DisplayPort, HDMI, GigE Vision, and USB3 IO. Basically, you can connect to most any image sensor and most any display standard with ready-made IP blocks that drop into your design, and the development boards give you the hardware to back it up.

Once you’re connected to your choice of sensors and displays, Lattice has assembled a powerful library of IP blocks to help you tame your video stream. These include image processing functions such as Defect Pixel Correction, HDR, Color Space Conversion, Black-level Correction, 2D Scaler, Gamma Correction, Overlay, AWB, De-Bayer, Video Frame Buffer, Color Filter, and Interpolation; bridging and aggregation functions including CSI-2/DSI D-PHY Receiver, CSI-2/DSI D-PHY Transmitter, Byte to Pixel Converter, Pixel to Byte Converter, FPD-LINK Receiver, and FPD-LINK Transmitter; and connectivity functions including DisplayPort Rx, DisplayPort Tx, GigE, and GigE vision. 

If figuring out how to connect the IP blocks you want to make an application is a little baffling, Lattice helps you get started with a rich set of reference designs that will cover a large gamut of typical applications. These include MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge, N to 1 CSI-2/DSI Duplicator, subLVDS to MIPI CSI-2 Image Sensor Bridge, N Input to 1 Output MIPI CSI-2 Camera Aggregator Bridge, 4:1 Image Aggregation, MIPI CSI-2 N to 1 Sideby-side Aggregation, 3D Depth Mapping, and ISP. If what you’re doing resembles one of those applications, your design is mostly done already (don’t tell your boss). 

Lattice’s tool support includes their legacy “Diamond” as well as their newer “Radiant” design tool suites – depending on your FPGA choice. And, if all these reference designs, IP, and tools still seem a little more than your team wants to bite off, Lattice has a range of design services available with experienced engineering talent that is well-versed in this type of design work – from their own Lattice Design Group, as well as third parties including Helion Engineering Vision, Tata Elxsi, and Bitec. These providers can do anything from developing a specialized IP block to implementing entire turnkey solutions.

The notion of the stack is to enable teams with a wide range of engineering skillsets and capabilities to take advantage of the remarkable capabilities that FPGAs bring to the embedded vision market. With the mVision stack, Lattice is largely uncontested at the ultra-low end of the power spectrum. None of the competitors such as Xilinx, Intel, and even Microchip (Microsemi) offer devices with the very-low power consumption of Lattice’s devices (although their FPGAs tend to be much larger and more capable). If your application is heavily power constrained, there don’t appear to be a lot of comparable FPGA alternatives.

It’s been interesting to watch Lattice re-define their mission and strategy after the failed acquisition a couple years ago. What seemed like bad news for the company at the time may have sparked a new wave of energy and innovation that will benefit the market overall. Lattice’s focus on small, inexpensive, low-power programmable logic technology now makes them unique in a market where demand for those qualities is likely to expand rapidly.