Panel: Multiple Gaffes Caused Fender-Bender Between Two Craft In Space
A newly-released Mishap Investigation Board report details a collision between two spacecraft in April last year, finding that not one but an entire series of cascading design flaws combined to cause the accident.
One of the craft was the Demonstration of Autonomous Rendezvous Technology (DART) asset by Orbital Sciences Corp., designed to move near and around another spacecraft without guidance from the ground. That automated rendezvous capability could be useful for the continued International Space Station program, an artificial moon of Earth that requires continual resupplies.
DART was to move near and around the Multiple Paths, Beyond-Line-of-Sight (MUBLCOM) satellite.
All went well on the mission for some time. The Pegasus XL rocket lofted the DART skyward from the Western Test Range at Vandenburg Air Force Base, Calif. And DART achieved orbit and a redezvous with MUBLCOM. So far, so good, except for some “irregularities” or “anomalies with the navigation system.”
But about eight hours into the flight, the DART was inhaling propellant like an SUV, trying to correct the spacecraft position and speed. After roughly 11 hours into the 24- hour mission, DART estimated that its propellant supply had run too low (which turned out to be erroneous), and it prepared to break off the rendezvous with MUBLCOM.
At that point, DART began maneuvers toward departure and retirement, bringing an early close to a $110 million program.
What wasn’t realized then is that DART already had collided with MUBLCOM three minutes and 49 seconds before initiating procedures for retirement.
While DART fully or partially met 11 of 27 mission objectives, it failed in its main mission objective, an expensive loss, and NASA declared a Class A mishap, meaning a loss of $1 million or more.
Some parts of the report on the extra-terrestrial fender-bender remain classified, but some findings of the mishap report are public.
One problem: because DART was intended to be autonomous instead of controlled from ground stations, it had no means to receive and execute uplinked commands. So the ground crews weren’t able to step in and take command of the situation.
Another problem is that DART was to shift from a GPS navigational system to another, the Advanced Video Guidance Sensor (AVGS) navigational system. DART did so, but only incompletely, because it missed a final navigational waypoint as it approached MUBLCOM.
That meant DART lacked information on the range to the MUBLCOM. DART could steer toward the other craft, but didn’t have accurate data on the closing gap between them.
Now, DART was equipped with a collision avoidance system, the system was in sound shape, and it did trigger a minute and 23 seconds before the collision.
But, despite all that, the collision avoidance system lacked the critical information on the exact distance and speed to MUBLCOM, and the system was unable to avoid the collision.
On the up side, though, the collision–rather than damaging MUBLCOM–pushed it into a slightly higher orbit, the board found.
If any one of the chain of causes leading up to the collision were to be removed, then it wouldn’t have occurred, according to a statement.
But one problem led to another, in a cascading series of glitches.
For example, with no accurate navigational data such as proper measurement of the distance to MUBLCOM, that meant DART used inordinate amounts of propellant in excessive thruster firings, which explains why the propellant supply ran out so soon.
And here’s another chain of events: DART detected that there was a growing disconnect between some data on navigation, and other data. So a programmed reset of the navigational processing was initiated. But in doing so, DART shifted from using AVGS to GPS, and the latter unfortunately wasn’t quite accurate, so there appeared a growing gap between what GPS was stating and what it should have reported. In turn, that meant that about every three minutes, the software went into yet another reset. And that, finally, led to creation of incorrect navigational data that caused the excessive thruster firings.
One troubling finding: the GPS inaccuracy was known previously. But a software change to fix the bug never was introduced in DART.
And here’s another one, in the GIGO (garbage in, garbage out) category: a software model that was run during pre-flight testing assumed that a receiver measured velocity perfectly. But in the real world, it didn’t.
Also, same system, different problem: design criteria assumed that the measurement of speed only had to be accurate to plus or minus two meters per second, when actually the design wasn’t able to handle that much error.
This all may seem sufficient to cause a major problem. But it wasn’t by itself sufficient to cause the repeated computational disconnects and software resets. Instead, there’s more.
DART had a program to look at any disconnect between measured position and speed, and estimated position and speed, and then weighted, or gave relative weight to, the two sets of numbers.
But the review board concluded that this logic function or reconciling the disconnects was set wrongly, so that once a disparity in the figures occurred and the software reset, the two numbers never could be reconciled into convergence.
The logic, introduced late in the program, was set to trust the estimated position and speed data inordinately, according to the board. And this change didn’t undergo proper testing and simulation to verify just what effects the weighting program would have.
To place all of this into a summary, the DART problem with persistent, inaccurate navigational information resulted from:
- Differences between the estimated and measured position of DART, causing the software to reset.
- Incorrect, erroneous velocity measure was thereby introduced.
- The navigational software design was overly sensitive to erroneous data.
- And there was an erroneous logic program to weigh the estimated versus the measured position data.
How bad was the effect of all this?
The bottom line is that at the time of the collision impact, when DART actually was moving toward MUBLCOM at 1.5 meters per second, DART programs thought that DART was 130 meters distant from MUBLCOM, and moving away at a speed of 0.3 meters per second.
One further problem is that the computer logic that determines the remaining amount of maneuvering fuel remaining in the tanks was faulty. When the DART determined that it had so little fuel left that it was time to withdraw and prepare for departure and retirement of the craft, DART actually had about 30 percent of the propellant fuel still in the tank.
The board also found fault with the way the DART program was arranged and the way the craft was designed.
As well, the panel suggested a series of moves to prevent similar problems from arising in future programs, such as using lessons learned from prior NASA programs, better systems integration procedures to spot design weaknesses, avoiding situations in which there can be a multiplicity of potential design interpretations, a process that permitted a failure to understand implications of potential effects that would be spawned by loss of functionality in one or more systems or subsystems, and more.
NASA released the board report.