NASA’s Voyager 1 spacecraft exited the Solar System’s heliosphere and entered interstellar space on August 25, 2012, after a multi-decade journey. Four of the spacecraft’s ten instruments are still reporting measurements back to Earth. The remaining instruments have been shut down to conserve energy as the power output of the spacecraft’s plutonium-powered radioisotope thermoelectric generation (RTG) continues to drop. On November 14, 2023, the Voyager team at the Jet Propulsion Laboratory (JPL) in Pasadena noticed that the spacecraft was sending garbled science and engineering data, although it still responded to commands. The spacecraft is currently 22.5 light-hours (15 billion miles) from Earth, so any troubleshooting involves a round-trip delay of nearly four days. Even with the obstacles of a 4-day latency and on-board computers that were designed and built half a century ago, JPL engineers were able to isolate the problem down to the chip level and effect a repair with a software patch. The first patch managed to restore the spacecraft’s engineering and status data. A subsequent patch or patches will restore the science instrument data.

 

The Voyager 1 spacecraft launched atop a Titan IIIE rocket on September 5, 1977. After more than four decades, the spacecraft is now in interstellar space. Image credit: NASA

The Voyager 1 spacecraft carries three dual-redundant computers: the Computer Command System (CCS), the Attitude Articulation Control System (AACS), and the Flight Data System (FDS). The FDS collects data from the spacecraft’s science instruments along with engineering data about the spacecraft’s health and status. It bundles this information, adds error-correcting codes so that the data can make it back to Earth, and then hands the coded information bundle to the telemetry modulation unit (TMU), which applies the modulation coding to the bundle and sends the modulated bundle to the spacecraft’s radio transmitter. For some reason, the TMU was sending a repeating pattern of ones and zeros instead of readable data. JPL’s Voyager team determined that the most likely problem was the FDS computer, not the TMU.

As a first troubleshooting step, the JPL team sent a “poke” command on March 1, 2024 that reset the program counter in the FDS computer. The command had an effect on the data stream but did not restart the flow of intelligible telemetry. On March 3, after the 2-day round trip latency, the Voyager mission team noticed that one section of the FDS bundle differed from the rest of the computer’s unreadable data stream. The new data was not properly formatted as normal telemetry. An engineer with NASA’s Deep Space Network managed to decode the new data. It was a memory dump from the FDS computer, including both executable code and data.

By analyzing the memory dump, JPL’s engineering team determined that about 3 percent of the FDS computer’s memory had been corrupted. The FDS memory stores 8K 16-bit words, or 16K bytes, and three percent of that capacity is approximately 4000 bits. Rounding up to the nearest power of two, the FDS memory had lost a 4-Kbit chunk. At this point, you might be thinking that the FDS is a dual-redundant computer, so JPL could just switch over to the backup. Unfortunately, the backup FDS computer’s memory failed more than forty years ago, on October 6, 1981.

There’s something quite special about the memory used by the FDS computers. While the CCS and AACS computers are based on older NASA computer designs that use plated-wire memory, the FDS computers needed much faster memory to achieve Voyager’s performance goals, so the FDS memory systems were built with CMOS SRAMs. I asked JPL for a description or part number of the failed SRAM and got a reply from Jeff Mellstrom, the Chief Engineer for JPL’s Astronomy and Physics Directorate. The failed device was one RCA 256×1-bit CD4061A CMOS SRAM. That one bad chip knocked out 256 words of FDS memory. It’s not possible to determine whether the SRAM simply wore out or was damaged by an energetic cosmic ray particle, and it doesn’t matter. No one would be sending out a repair mission. The fix, if one was possible, had to be sent by radio.

The JPL engineering team reworked the code in the FDS that formats the engineering data, breaking it into pieces that would fit into the unaffected memory addresses and adding jump instructions to skip over the affected memory. This work had to be done by hand. According to Mellstron, “…the assembly language is documented, but there no longer are development tools, hardware testbeds, or software emulators. To implement the code relocation as quickly as possible, we proceeded without developing a bit-level simulation of the processor. Because of these limitations, it was not possible to test the code modifications before uplinking to the spacecraft. Verification of the code was by inspection only.”

That’s some full-on, old-style, gutsy engineering. We’re not talking about a WiFi firmware update here. We’re talking about beaming a software patch 22.5 light-hours into the ether, beyond the edge of the Solar System, using NASA’s Deep Space Network. Incredibly, it worked. On April 22, Voyager 1 started sending intelligible engineering telemetry back to Earth. The team plans to restore the science data stream in the same way.

However, that’s not the end of this story. The RCA CD4061A 256×1-bit CMOS SRAM has an interesting place in semiconductor history. It’s part of RCA’s venerable CD4000 family of CMOS logic ICs. RCA was the leading early advocate for CMOS. (See “A Brief History of the MOS transistor, Part 5: RCA – The Persistent CMOS Contrarian.”) After years of work on CMOS ICs starting in 1963, RCA finally announced the first 15 members of the CD4000A IC family in 1968. By 1969, there were fifteen CD4000A family members. RCA was the sole source for CD4000A chips that year. By the end of 1972, when RCA devoted the entire December 1972/January 1973 issue of “RCA Engineer” to CMOS technology, there were more than 60 devices in the CD4000A family with multiple alternate sources.

In that same issue of “RCA Engineer,” an article titled “COS/MOS Standard-parts Line Comes of Age” listed all the members of the CD4000A family. (“COS/MOS” was the trademarked name RCA used for CMOS.) This article did not list the CD4061A SRAM, but it did list a 256×1-bit SRAM with a “developmental” part number: the TA6335. During that period, RCA was designing custom CMOS parts under contract for the U.S. Air Force and for NASA, so the TA6335 might have been one of these contracted devices, or it may have been developed by RCA for its own purposes. Whatever the reason, the device was introduced as the CD4061A SRAM in 1974 and appeared in the 1975 COS/MOS IC catalog.

 

Die shot of the RCA CD4061A 256×1-bit CMOS SRAM, still bearing the development number “6335” along the right-hand side. Image credit: Steve Emery, ChipScapes.com

RCA’s CD4061A SRAM was pin-compatible with Intel’s 1101 SRAM, which had been introduced in 1969. However, the CMOS CD4061A drew far less power than the Intel 1101. Although nowhere near as famous or as successful as Intel’s 1103 DRAM, the 1101 was Intel’s first successful MOS IC, so making the CD4061A pin compatible with the 1101 was a pretty good idea.

A report appearing in RCA’s “High Reliability Integrated Circuits” data book in 1982 stated that Voyagers 1 and 2 contained 10,346 RCA CMOS ICs between the two spacecraft. That works out to 5173 CMOS chips per spacecraft. Each of the dual-redundant FDS computers in each spacecraft needed 512 CD4016A SRAMs to create the 8Kword memory, which is 1024 CMOS SRAM chips per spacecraft. According to the reliability report, just two of the RCA CMOS ICs out of the 10,346 total number in both Voyager spacecraft had failed after the first 52 months in space. We know from JPL that one of the Voyager 1 spacecraft’s FDS memories failed in 1981, so at least one of the two reported CMOS chip failures was associated with that memory loss. Possibly both. The fact that these very early RCA CMOS SRAMs managed to get Voyager 1 out past the edge of the Solar System in working condition is a real testament to the design and manufacture of the chips and the design and fabrication of the Voyager FDS computer systems. The fact that the JPL engineering team could troubleshoot and diagnose a problem and effect a repair after nearly 50 years and from 15 billion miles away is a testament to the dedication of that team. Sort of makes you proud to be an engineer, doesn’t it?

Epilog

The 10,346 CMOS ICs in these two spacecraft are part of the dwindling number of artifacts that represent the innovation factory that was once RCA’s Solid State Division. During the 1960s, after David Sarnoff’s son Robert Sarnoff took over as RCA’s president, the company veered away from its role as a technology leader and became a conglomerate by making a diverse set of corporate acquisitions, including Hertz rental cars, Banquet frozen foods, Coronet carpets, Random House book publishing, and Gibson greeting cards. Radio Corporation of America (RCA) abandoned its original name and simply became “RCA,” while investors nicknamed the new conglomerate “Rugs, Chickens, and Automobiles” to better reflect the company’s new posture.

RCA did not survive the Robert Sarnoff era for long. GE announced its intent to reacquire RCA in late 1985. It had created RCA as an electronic technology spinout at the request of the U.S. government back in 1919. After the acquisition was completed in 1986, GE Solid State absorbed RCA’s Solid State Division along with old-time semiconductor maker Intersil, Inc. However, GE’s chair Jack Welch (“Neutron Jack”) had little appreciation for the semiconductor business with its wildly fluctuating financials and sold GE Solid State to Harris Corp in 1988. Harris sold GE Solid State’s logic business, including RCA’s CD4000 CMOS logic family, to Texas Instruments (TI) in 1998, although Harris continued to make radiation-hardened versions of RCA’s CD4000B logic family. (The “B” is for “buffered.”) Harris spun out the rest of its semiconductor business as a newly reconstituted Intersil Corp in 1999, which was subsequently absorbed by Renesas in 2017.

You can still get CD4000B devices from TI. STMicroelectronics lists radiation hardened CD4000B devices as current products. Rochester Electronics – a company that specializes in keeping certain obsolete semiconductors available through authorized old stock, die banks, and licensed reverse or re-engineering – seems to have a few CD4000A devices originally made by RCA and Harris in stock, available through DigiKey. However, you’ll be hard pressed to find any CD4061A SRAMs still on the shelf. I didn’t find any, although I did locate a cache of bare CD4061A die, in possession of a friend.

Meanwhile, Voyagers 1 and 2 continue their mission to sail beyond the Solar System and beyond. If you want to check the Voyagers’ status, NASA maintains a real-time update page here.

References

H. Weisberg, “MOS – an RCA pioneered technology; COS/MOS – RCA’s thrust in digital logic,” RCA Engineer, Dec/Jan 1972/1973, pp 5-7

R. Heuner, “COS/MOS standard-parts line comes of age,” RCA Engineer, Dec/Jan 1972/1973, pp 14-19

T. G. Athanas, “Development of COS/MOS technology,” RCA Engineer, Dec/Jan 1972/1973, pp 26-30

James E. Tomayko, Computers in Spaceflight: The NASA Experience, NASA History Office, July 2005

“High Reliability Integrated Circuits,” RCA, 1982

• 
• 
• 
• 
•