SpaceX completes Falcon 9 Failure Investigation, Missions to resume Sunday from Vandenberg

SpaceX concluded an exhaustive investigation into the cause of the dramatic explosion of a Falcon 9 rocket during a pre-launch test on September 1 and is now looking forward to returning to launch operations as early as Sunday with the first of seven missions dedicated to deploying Iridium’s next generation of low-orbiting communications satellites.

The California-based company had established a regular flight rate of around one mission per month before being stopped in its tracks on September 1, 2016 when Falcon 9 suddenly exploded while being tanked for a routine engine firing test two days prior to the planned launch. The mishap claimed the loss of the $195 million AMOS-6 communications satellite which had been placed atop the rocket for the static fire test.

Working through an extensive fault-tree analysis, SpaceX concluded that the mishap was triggered by a complex mechanism that caused the failure of a high-pressure helium tank on the rocket’s second stage. Corrective actions are being implemented and SpaceX has identified future design changes to make Falcon 9 safer.

SpaceX’s return to flight mission is planned to lift off from SLC-4E at Vandenberg Air Force Base at 18:28 UTC on Sunday, provided a Static Fire test occurring in the coming days goes off without a hitch.

Falcon 9 will be tasked with taking ten Iridium-NEXT satellites to an orbit 750 Kilometers in altitude, the first of seven Iridium launches lined up on SpaceX’s manifest to deploy the full constellation of LEO communications satellites and rejuvenate Iridium’s space segment which currently consists of antiquated spacecraft launched in the late 90s and early 2000s.

It is not yet known when SpaceX can resume flights from Cape Canaveral as the company works to fully activate Launch Complex 39A at the Kennedy Space Center that will serve as the primary east coast launch pad for Falcon 9 until Space Launch Complex 40 can be repaired after receiving extensive damage in the September 1 mishap. First on the Cape manifest for Falcon 9 is the launch of the EchoStar 23 communications satellites which can be expected no-earlier than January 15 and will likely be followed by the Dragon SpX-10 resupply mission to the Space Station.

On September 1, just over five minutes from a test firing of Falcon’s nine Merlin 1D first stage engines, the second stage of the rocket suddenly blew apart in a high-speed conflagration without giving any prior signs of trouble to the Launch Team on station to oversee the test. The 70-meter tall rocket and its satellite payload then collapsed onto the pad structure in a large blaze of fire.

SpaceX halted all launch operations at that point and headed into an exhaustive investigation to piece together what could cause the rocket to blow up while still in the process of being filled with propellants. According to SpaceX CEO Elon Musk, this case proved out particularly difficult due to the extremely rapid timeline of the failure with only 93 milliseconds from the first sign of trouble to the loss of telemetry data from the rocket.

After an initial investigation into all possible causes internal and external to the rocket, SpaceX quickly zeroed in on a breach in the second stage’s helium pressurization system.

“Investigators scoured more than 3,000 channels of video and telemetry data,” SpaceX said in Monday’s statement. “Because the failure occurred on the ground, investigators were also able to review umbilical data, ground-based video, and physical debris.” To collect more data, SpaceX rigged up a test system at the company’s McGregor site where a number of tanks were blown up as the company attempted to re-create the conditions of the failure to piece together what factors contributed to the explosion.

Engineers found a complex interplay at work during the rapid 2nd stage tanking sequence between the Liquid Oxygen flowing into the tank, the Composite Overwrapped Pressure Vessels submerged in the oxidizer and the cold helium being forced into the vessels. Investigators “concluded that one of the three composite overwrapped pressure vessels (COPVs) inside the second stage liquid oxygen (LOX) tank failed.”

Falcon 9 uses high-pressure helium to pressurize its propellant tanks during flight. The gas is stored at extremely high-pressure in the COPVs which consist of an aluminum liner wrapped in a carbon composite material that gives the tank the mechanical strength needed to store the helium gas at nearly 400 bar.

“Specifically, the investigation team concluded the failure was likely due to the accumulation of oxygen between the COPV liner and overwrap in a void or a buckle in the liner, leading to ignition and the subsequent failure of the COPV.”

SpaceX noted that the recovered COPVs showed buckles in their liners and investigators concluded that sub-cooled Liquid Oxygen can pool in these buckles under the carbon overwrap. The loading temperature of the helium on the other side of the thin liner was shown to be cold enough to create solid oxygen which then became trapped – creating extremely volatile conditions with solid oxygen oxidizer trapped in place and carbon-fiber material acting as a fuel source.

Ignited through friction with the fibers or through pressure from the expanding liner, the oxygen combusted with the carbon material, causing the COPV to fail in a spectacular way.

Corrective actions implemented by SpaceX begin by “changing the COPV configuration to allow warmer temperature helium to be loaded, as well as returning helium loading operations to a prior flight proven configuration.” Loading the helium well ahead of the Liquid Oxygen allows the COPVs to enter a stable state which avoids trapped oxygen getting compressed when the liner begins pressing against the composite overwrap. LOX loading procedures will remain unchanged and SpaceX did not comment on possible performance penalties when using the more conservative helium temperature.

In the long-term, SpaceX is working to develop COPV manufacturing techniques that prevent buckles altogether which will allow for faster loading operations once these changes are fully implemented.

SpaceX held the overall responsibility for the investigation with oversight provided by the FAA as the primary federal licensing body. NASA, the U.S. Air Force, the National Transportation Safety Board and several industry experts also participated in the investigation with full access to the results to build confidence in the changes implemented to return the Falcon 9 launch system to flight.

While the investigation was in its final stages, SpaceX returned to regular acceptance testing of rocket stages at the McGregor facility and shipped two Falcon 9 launch vehicles to their respective launch sites on both coasts of the United States. SpaceX was initially going for a return to flight before the end of 2016 but that timeline could not be kept and the company opted to take additional time to finish the investigation and get all the required approvals.

The ten Iridium-NEXT satellites waited out the Falcon 9 stand-down at the Vandenberg launch site as they had already entered final launch processing when the September 1 mishap occurred. Re-entering their processing flow, the satellites were loaded with propellants and integrated around two cylindrical dispenser segments that were then stacked and encapsulated in Falcon’s protective payload fairing.

Falcon 9 will make its return to the SLC-4E launch pad later this week for a Static Fire test which needs to be checked off before the rocket is taken back to the hangar to receive its payload. Liftoff is planned for 18:28:07 UTC on Sunday – an instantaneous launch opportunity that must be hit on the mark to lift the satellites to their orbital destination.