“While the camera does not express the soul, perhaps a photograph can!” – Ansel Adams

Highway signs flicker. We just don’t notice it. 

This was one of the takeaways from a long discussion about camera sensors for automotive applications. Cars have cameras in them now, some outward facing and some inward facing. They’re used for different purposes (more on that in a moment), but the outward-facing ones have a slew of problems to deal with that I’d never considered. One of them is the nasty flickering of highway signs. 

Turns out, most electronic signs by major roads use LEDs, and contrary to all appearances, those LEDs aren’t really on all the time. They’re strobed at about 90 Hz, partly to save energy and manage brightness. That flickering is invisible to us; our eyes have enough persistence that we perceive the lights as solidly on. (Old-school fluorescent lights work much the same way.) 

This isn’t a problem for us humans, but it is if you’re a camera. Your average CMOS image sensor doesn’t have any visual persistence, so it will “see” the LEDs switch on and off several times per second. Depending on the frame rate and shutter speed of your camera, it might see the LEDs on all the time, off all the time, or some pseudo-random combination of the two. 

This is similar to the “helicopter effect,” where the rotational speed of the aircraft rotor happens to match the shutter speed of the film or video camera, making it look like the rotor is stationary even though the craft is flying. Your great-grandparents might have recognized this as a zoetrope. 

None of this amounts to more than a slight nuisance to videographers – unless you’re designing automotive safety systems. Then it’s a real problem. It isn’t just limited to exotic Level 5 self-driving systems, either. Even midrange safety systems that detect stop signs or speed limits can be fooled. What’s a poor designer to do? 

“Buy our image sensor,” says OmniVision, predictably enough. The company just announced its newest OX03C10 in-car camera chip that includes something called LFM: LED Flicker Mitigation. An image sensor’s exposure time is shorter than the illumination time of a roadside LED, so the ’C10 drags out exposure time to 11ms, takes multiple exposures, and does some internal magic that the company hesitates to describe. The result is that the image presented to downstream logic is flicker-free. Just like it looks to us.

OmniVision credits its stacked-die assembly for the improvement. Image sensors and digital logic want to be built on different semiconductor processes, so that’s what the company does. 

Headlights and illuminated billboards flicker, too, but these present a different problem. Recognition isn’t the issue; it’s displaying the video feed. If you pipe the incoming video to the car’s in-dash display, it will reproduce the flickering, which is ugly and distracting. It’s also weird, since we don’t perceive the original source that way. There must be something wrong with my car! So, systems have to filter out the flickering from all sources and present a smooth, but untrue, image of the world. The ’C10 takes care of that before the video leaves the chip. 

Looking the other way, some cars also have cameras pointed at the driver, usually to detect drowsy or distracted drivers. High-end cars have had this feature for the last ten years or so. The trick here is (a) to get a clear image of the driver, and (b) not to distract him/her in the process. Driver-monitoring cameras need to be small and discrete (preferably invisible), and they have to work in total darkness. Naturally, OmniVision has a new chip for this application, too. 

Since drivers don’t move around in their seat very fast, the DMS (driver monitoring system) camera can be pretty slow. Most use a rolling shutter, where each line of the image sensor is raster scanned. It takes a while to scan the whole sensor, but it’s plenty fast enough to capture a reasonable image. The trouble with slow scanning is that the light source (generally an infrared LED) needs to be illuminated the whole time, and that consumes more energy than the image sensor does. So, it makes sense from a power perspective to scan faster and turn the light off quickly.  

OmniVision’s new OVM9284 uses a global shutter, snapping the entire image in a few hundred microseconds. Each pixel is backed up with short-term storage (really just a capacitor) so that the image can be scanned out serially before it’s lost. 

OmniVision also replaced the traditional 850nm wavelength LED with a 940nm light source, on the theory that one is visible to the naked eye and one isn’t. It would be counterproductive to have a distracting red glow coming from your DMS camera while it’s monitoring you for distractions. 

Now, if we could just point those cameras the other way and make sure all the other drivers aren’t distracted. That would be a feature worth having.