Causes of Software Failure

Most people who use computers have run into a program that stops responding or closes without warning. Often the cost is small: a few minutes of frustration and a window to reopen. This article is about the other end of that range, the failures that cost far more than an annoyance.

As you read the cases that follow, keep one question in mind: at what point in the chain from a programmer's decision to a physical outcome did things go wrong? That question separates a useful account of a failure from a simple list of disasters. The source material also points to a pattern worth noticing early. The cases get more serious as the work assigned to the computer gets more important.

At the mild end of the spectrum, a failing program is an annoyance. On a personal computer, a failure can still cause lost work or lost money. When a computer is given more important tasks, the consequences of its failure become proportionately more serious. In the most severe cases, they include loss of life.

The diagram shows the three levels of consequence the source describes. The same underlying kind of failure can land at any of these levels. What moves it up the ladder is the importance of what the computer is doing when the failure happens, not the failure's technical form alone.

In September 1999, the Mars Climate Orbiter burned up in the Martian atmosphere. The source gives the cause in a single sentence: data expressed in English units of measurement, such as feet and pounds, was entered into a computer program designed to use metric units, such as centimeters and grams.

Nothing in the program was simply wrong in the sense of a line of code that crashes. The program did what it was built to do. The numbers it received did not mean what it assumed they meant. The failure lies in the gap between the unit system the data was written in and the unit system the program read it as.

The diagram shows the two sides of the mismatch. The data was written in one system, and the program read it as if it were written in the other. The source lists this case among problems caused by incorrect assumptions as well as by unit mismatches, because the mismatch is itself an unchecked assumption about what the numbers mean.

A few months after the Orbiter was lost, the Mars Polar Lander probably crashed. Its software was supposed to detect the bump when the spacecraft touched down and then turn off the engines. The source reports that deploying the landing gear might have jarred the spacecraft enough to activate that detection step early.

The program then turned off the engines while the spacecraft was still high above the ground. The source describes the unpowered craft as one that would have fallen to the surface. The program's assumption was that a touchdown signal could only arrive at the moment of touchdown. A single triggering signal arriving at another time broke that assumption.

The source names three domains where failures directly affect people's health, money, or safety: medical devices, financial systems, and aerospace applications. Software failures have not stopped with the older missions. The source also cites computer software error as a contributor to the Northeast Blackout of 2003, one of the largest power outages in history.

The source lists further incidents. In 2006, the Airbus A380 was delayed by software incompatibility problems, at a cost the source estimates may have reached billions of dollars. In 2007, a software problem grounded thousands of planes at the Los Angeles International Airport. On May 6, 2010, a flaw in an automatic trading program apparently resulted in a 1000-point drop in the Dow Jones Industrial Average. A 1990 disruption of AT&T's long-distance telephone service throughout the United States was caused when a newly loaded computer program proved to contain a bug.

The Orbiter case shows something easy to miss. The program in that case was not broken in the way a learner might first imagine. It executed correctly according to its design. The failure appeared because the data arriving at the program meant something different from what the program assumed.

This means a failure is best understood as a chain. It starts with a decision about how the system should work, passes through data entering the system, and ends in a physical outcome. A failure can enter the chain at any link. The question to ask is at which link the error was introduced, not only which line of code is involved.

The diagram is a general model rather than a claim about a single cause. In the Orbiter case, the break occurred at the interpretation step, where metric assumptions met English-unit values. In the Polar Lander case, the break occurred at the same interpretation step, where the program trusted a signal to mean touchdown when it did not.