SCE to Aux
52 years ago, on November 14th, 1969, John Aaron saved the Apollo 12 mission from complete disaster. Five months later, he would do it again, when he helped Ken Mattingly and many other NASA engineers come up with a low-voltage start-up sequence for Apollo 13’s dead command module. You may be already familiar with the events of Apollo 13 from the excellent 1995 Ron Howard film (and if you aren’t, please add it to your “Must Watch” list with all possible haste). The Apollo 12 incident is not as well-known, but it was perhaps more astounding than Apollo 13, because John Aaron, once a farmboy who had wanted to be a cattle rancher, confronted the crisis entirely alone, and he had less than two minutes to fix it.
On Friday, November 14th, John Aaron was sitting in the EECOM chair inside Mission Control at Cape Kennedy, Florida. EECOM stood for Electrical, Environmental, and Consumables Manager, which was another way of saying he was in charge of the systems keeping the astronauts alive. The Apollo 12 launch was originally scheduled for Thursday, November 13th, but thunderstorms delayed the launch. The launch had a narrow two-day window because part of Apollo 12's mission was to land on a very specific spot on the moon. Two years earlier, the Surveyor 3 probe had landed on the moon to take pictures. NASA was pretty keen on the idea that astronauts could bring pieces of the probe back to earth so they could study how certain man-made materials were affected by exposure to the harsh lunar environment. This was the last day they could launch or they’d miss the window to get to the Surveyor site. On Friday, the thunderstorms had moved through, leaving behind what looked like just clouds and rain. With President Nixon watching, the launch proceeded flawlessly, and on Friday November 14th at 11:22 am, the Saturn V rocket lifted off.
36 seconds later, pants-wetting disaster struck. As the Saturn V punched its way through a dark raincloud, the 363-foot metal rocket and the hot ionized gasses in its exhaust plume acted like a lightning rod, and the raincloud obliged by frying the bajeebers out of it with the requisite 1.21 gigawatts... not once, but twice. The first bolt knocked out all three fuel cells that the command module was using as its primary power source, and the second bolt knocked out the "8-ball" attitude indicator, which told the crew if the rocket was flying in the right direction. There was no live video feed from the launchpad, so the crew and ground control didn't know about the lightning strikes.
Commander Pete Conrad, sitting in the left-hand seat in the Apollo capsule, had an inkling of what happened, as he'd seen a flash of light and felt the rocket shudder. However, the more pressing problem confronting him was the Command Module's control panel was lit up with angry warning lights like a Christmas tree dipped in gasoline. A few seconds later, the telemetry signals from the ground blinked out. Every possible display or indicator that could tell him where the rocket was going was either dead or flashing angry red warning lights.
Conrad put his hand on the abort handle while Dick Gordon, the Command Module Pilot, worked his way through the warning lights. The abort handle would activate the Launch Escape System (LES), a smaller rocket on top of the capsule that was supposed to disconnect the capsule from the Saturn V and get it far enough away so it could splash down in the ocean, hopefully with three astronauts still alive inside it. But the escape system had never been tested with astronauts actually inside the capsule, and without any telemetry from the ground and a dead instrument panel, Conrad couldn't tell if the Saturn V was still heading in a direction that would allow the astronauts to escape. A decision had to be made quickly, because in another two minutes the rocket would be too high to use the escape system. He should have already aborted by now, but it still felt like the rocket was on course. With his fingers still curled around the abort handle, Conrad called back down to mission control, "Okay, we just lost the platform, gang. I don't know what happened here. We had everything in the world drop out."
Mission Control didn't know what was going on, either. They were staring at the same warning lights the crew was staring at, and it looked like the entire electrical system had gone haywire. It hadn't gone completely dead, but for the purpose of guiding the spacecraft, it might as well be.
Alan Bean, seated on Dick Gordon's right in the capsule, was just as confused. He could see that the capsule instruments appeared to have switched over to backup battery power, which should have happened if Conrad had pulled the abort handle, but he hadn't felt the "kick" from the escape rocket (at seventeen G's, or seventeen times the pull of Earth's gravity, it would have been quite a kick... the highest G roller coaster on Earth can do just above six G's). The backup batteries in the capsule were only supposed to provide power during re-entry. They would only last a few hours, so even if they did manage to get up into orbit, with no way to recharge the batteries or operate their instruments, they could wind up in a cold, dead capsule with no way to get back down to Earth alive. Bean knew that the instrument panel in front of him needed 28 volts of AC (alternating current) to operate properly, but the system wasn't entirely dead. Bean called out, "I've got AC."
Conrad said, "Maybe it's just the indicator. What do you got on the main bus?"
Bean replied, "Main bus is—the volt indicated is 24 volts."
If there was a short in the electrical system, Conrad was trying to nail down where it might be. If it was a problem between the indicators and the rest of the system, then the rest of the rocket might be OK and still on course. But they didn't have a lot of time to hunt down a short and fix it while the largest non-nuclear explosive device humankind had ever built exploded behind them. If they were going to use the LES, they only had about two minutes left to use it.
That decision fell to Gerry Griffin, the flight director. This was his first launch as flight director, and less than a minute after launch he was faced with a crisis that could kill three astronauts and put an end to the entire Apollo program. A little over a year before, NASA had lost Gus Grissom, Ed White, and Roger Chaffee in the Apollo 1 fire. Losing another crew could sour the public on the space program and cut off funding from Congress. Since the problem appeared to be with the electrical system, Griffin turned to John Aaron, who was sitting in the EECOM chair, and asked him, "EECOM, what do you see?"
John Aaron had not grown up dreaming of launching rockets, flying spaceships, or landing on the moon. He grew up on a cattle ranch in rural Oklahoma. His high school was so small, there were only nine people in his graduating class. When faced with what to do about finding a career, he figured he’d just do what his family had always done: raise cattle. However, he needed a side-job to provide enough income to start his herd. The second-youngest in a large family, most of his seven sisters had gotten into the education profession. John figured he’d do the same, pick up a physics and math degree, and then teach math until he could afford some Herefords.
After a year at Bethany Nazarene College, he transferred to Southwestern Oklahoma State University. During his senior year, he ran into a problem: he had become so involved with math and physics, he had not bothered to complete his education courses. It would take him another year of school to finish them, but he was broke. Fortunately, a friend who had graduated a year ahead of him offered another option: NASA was looking to hire people with science degrees for the space program. John sent in an application on a whim, guessing he might at least get an interview. Instead, NASA sent him a job offer for $6,700 a year, more money than the farmboy had ever seen in his life. He figured, well, he could do that for a few years, and then he’d have enough money to buy some Herefords and start his cattle ranch.
Five years later, less than a minute after the launch of Apollo 12, Gerry Griffin turned to John Aaron and asked, “EECOM, what do you see?” This is where events get legendary. No one else in the room had ever seen this kind of scenario before, and we're talking about a lot of NASA scientists who just spent most of a decade figuring out every possible way you can blow up a rocket or kill an astronaut. As John Aaron looked down at his screen, he saw gibberish. Not a blank screen, or a screen full of zeroes, but bogus numbers that didn't make any sense coming from a 28-volt electrical system. However, there was something oddly familiar about this pattern of numbers. He'd seen this pattern before.
A year before, John Aaron was sitting in the EECOM seat during a simulated launch and for a few seconds he'd seen a similar pattern of numbers flash on his screen. Just a few seconds. He'd never seen anything like that come up before, and it stuck in his brain. He knew the electrical system backwards and forwards, but couldn't figure out what sort of electrical problem would give you gibberish numbers instead of no numbers at all. The morning after the simulated launch, Aaron got a hardcopy of the test data, and then talked to the controllers at Cape Kennedy. Someone grudgingly admitted that one of the test controllers had accidentally lowered the voltage, and for a few seconds, only one fuel cell was supplying power before the mistake was corrected. That explained why the numbers on Aaron's screen had changed, but not the pattern of the numbers. So he enlisted the aid of an instrumentation engineer, and was able to track the problem back to something called the Signal Conditioning Equipment (SCE).
The SCE was a box of electronics that took in all the signals from the sensors throughout the rocket and the capsule, which could come from a variety of sources with either alternating current (AC) or direct current (DC) and different voltages, and adjusted them into a much more manageable range of five DC volts for the instrument panel. The SCE normally required 28 volts to operate correctly, but if there was a sudden loss of power to the space vehicle, it was designed to automatically switch over to an auxiliary backup power source, such as a backup battery. The backup batteries operated at lower voltage, but the SCE was designed to compensate for this. If the automatic switch didn't work, there was also a manual switch inside the capsule that would do this as well, labeled "SCE to AUX". The SCE was the source of the wonky numbers from the simulated launch. With only one fuel cell providing power, the voltage dropped, but not far enough to automatically switch the SCE to auxiliary power. Flip the switch, though, and the SCE should work just fine on low-voltage settings.
Inside Mission Control, Griffin waited for Aaron to tell him that whatever was wrong with the electrical system, it was beyond repair, and the best chance they had to get the astronauts back home alive was to abort. Time was running out, because in about 90 seconds, the Saturn V’s second stage would ignite and the rocket would be too high to use the launch escape system.
But that's not the call that came from EECOM.
Aaron turned his steel-gray eyes toward Griffin, and calmly said, "Flight, try SCE to Aux."
Griffin had absolutely no clue what Aaron had just said, and asked, "Say again. SCE to off?"
Aaron clarified, "Aux."
Griffin was still confused. "SCE to Aux?"
Aaron clarified again, "Auxiliary, Flight."
Griffin still had no idea what Aaron was talking about, but he trusted that EECOM was the expert on anything involving the electrical system. He passed the command to Gerry Carr, sitting in the Capsule Communications (CAPCOM) seat. Carr, another Apollo astronaut, was the only one in the room allowed to talk to the astronauts inside the capsule. Carr also had no idea what Aaron was talking about, but passed the command along to the crew.
Carr: "Apollo 12, Houston. Try SCE to Auxiliary. Over."
Conrad: "NCE to Auxiliary. What the hell is that? "
Carr spelled it out: "Ess See Eee. Ess See Eee to Auxiliary."
Conrad had no idea what the command meant or where to find the switch. Gordon, sitting next to him, was still trying to nail down what was wrong with the fuel cells and electrical buses. The SCE subsystem was so obscure that the flight director and three Apollo astronauts had no idea what it did or where the switch was.
Fortunately, Alan Bean, the third astronaut in the capsule, knew exactly where that switch was. It was right above his shoulder. While Conrad was trying to talk to CAPCOM and Gordon was chasing down warning lights, Bean reached out and flipped the SCE to AUX switch to AUX. There are several accounts out there that state Bean knew about this switch because it had come up on a previous training simulation, and at least one claim that Bean had been in the same launch simulation where Aaron had seen the strange pattern of signals. But these conflicting accounts are likely dramatic fabrications. According to Aaron, the SCE switch was so obscure that it wasn't part of the launch testing, and there were no training simulations that involved switching it to AUX. The only reason Aaron knew about it was because he saw some gibberish on a screen for a few seconds and needed to know what caused it. Even after studying the SCE in detail, it never occurred to Aaron or any other NASA engineer that the SCE might fail to switch over to Auxiliary if the main power to the instrument panel were disrupted. On that particular day, there were only two people involved with that launch who knew where the SCE to AUX switch was. One of them was sitting in the EECOM chair at Mission Control. The other one was sitting in the right-hand seat in the capsule. Alan Bean knew his instrument panel so thoroughly that he instantly remembered where the switch was and activated it.
Within seconds, telemetry came back up and everyone could see, in the capsule and back at mission control, that the rocket was still on-course. There were still angry warning lights flashing everywhere, but the instrument panel was working and the numbers were making sense. Mission control told the crew to prepare for the next stage of the rocket to ignite, while Aaron figured out that all three fuel cells in the service module had been knocked offline, and told CAPCOM to direct the crew to turn them back on. Alan Bean reset the fuel cells. The first two came back online immediately, and the third came back a few seconds later. There was still some technical stuff to work out about which fuel cells and batteries switched to which buses, but the command module was back at full power, the second stage ignited smoothly, and the telemetry looked good. As the second stage boosted them into Earth orbit, Conrad started laughing, and laughed all the way up into Earth orbit, because he couldn't believe that he was still alive and on his way to the moon.
Back at Mission Control, tension had eased but the NASA engineers were not laughing. On the way up to orbit, Conrad mentioned the lightning strike and jokingly recommended more all-weather testing. Eye-witness reports from the launchpad confirmed the lightning strikes. The engineers on the ground realized there were two big things that could still kill the three Apollo astronauts. The first concern was the lightning strikes could have damaged the electrical system. NASA engineers had considered lightning strikes on the launch pad as a risk, but they'd never anticipated a strike while the rocket was in flight, and they didn't have any procedures for it. If the electrical system was damaged, then they needed to know about it before the command module started its burn towards the moon and left Earth orbit. So Mission Control added another orbit around the Earth to give John Aaron and the crew more time to develop new procedures for checking out the electrical system, finish the testing, and make sure they were confident that the system was working well enough to continue to the moon. Chris Kraft, the director of Flight Operations and thus Griffin's boss, came to Griffin at this point and told him, "Young man, don’t feel like we have to go to the Moon today, but on the other hand if you and the other systems people here can quickly check this vehicle out and you feel comfortable with how to do that then we’re okay to go, but don’t feel you have to be pressured to go to the Moon today after what happened. We don’t have to go to the Moon today." So, you know...no pressure! Imagine if your boss put THAT decision in your hands.
The second concern was the lightning strikes might have damaged the parachutes in the nose of the command module. There was no way for the crew or mission control to check the parachutes or verify if the explosive charges to deploy them would still work. But if the parachutes were going to fail, the crew was going to die during re-entry regardless of whether or not they went to the moon. After John Aaron cleared the electrical system, Griffin made the call: send Apollo 12 to the moon. Mission control decided they just weren't going to mention anything about the parachutes to the crew.
The rest of the mission proceeded smoothly. Instead of Neil Armstrong's nail-biting boulder-dodging fuel-guzzling landing on Apollo 11, Conrad and Bean landed the lunar module right on target, within walking distance to the Surveyor 3 probe. They stayed on the lunar surface for 31 hours and brought back 75 pounds of rock along with pieces of the camera from Surveyor 3. The journey back to Earth went just as smoothly. The capsule re-entered the Earth's atmosphere, and the parachutes worked perfectly. The only injury to the crew happened during splashdown, when a handheld camera was dislodged and struck Alan Bean on the forehead, briefly rendering him unconscious and giving him a concussion.
There are several ways to consider what happened during those terrifying two minutes and how things turned out the way they did. What are the odds of not one but two lightning strikes, that the steely-eyed missile man sitting in the EECOM chair was John Aaron, that he remembered a weird screen from a year ago that he'd only seen for a few seconds, that he was curious enough during his own free time that he tracked down where those funny numbers came from, that the designers of the SCE included a low-voltage setting, that Alan Bean remembered exactly where that switch was? It might be tempting to use the word "miraculous" here, but that's not really the right word for it. A huge amount of science, engineering, planning, and incredible hard work went into putting the right equipment and the right people in place for that launch, all to accomplish an entirely human endeavor. It was human beings who decided to go to the moon. It was human beings who designed and built a rocket and spacecraft that could get them there and back to Earth. For one tiny infinitesimal blink during the immeasurable stretch of time, human beings from the planet Earth scratched a white-hot streak of light across the infinite darkness of the universe. Even if the only reason human beings did this was just to know that we could do it...can you imagine anything more completely astounding, inspiring, and just jaw-dropping awesome?