Writing Robust Programs

Most people who use computers have run into a program that stops responding or closes without warning. Often the cost is small: a few minutes of frustration and a window to reopen. This article is about the other end of that range, the failures that cost far more than an annoyance. The aim is to build a habit of mind that will be useful whenever you write or rely on software.

As you read the examples that follow, keep one question in mind: at what point in the chain from a programmer's decision to a physical outcome did things go wrong? That question separates a useful account of a failure from a simple list of disasters.

The source describes a spectrum of consequences. At the mild end, a failing program is an annoyance. On a personal computer, a failure can still cause lost work or lost money. When a computer is given more important tasks, the consequences of its failure become proportionately more serious, and in the most severe cases they include loss of life.

The diagram shows the escalation the source describes. Each step up the ladder corresponds to a more important job given to the computer. The levels are a teaching device drawn from the source's own examples, not a formal classification of software failures.

In September 1999, the Mars Climate Orbiter burned up in the Martian atmosphere. The source explains the cause in a single sentence: data expressed in English units of measurement, such as feet and pounds, was entered into a computer program designed to use metric units, such as centimeters and grams.

Notice what did not go wrong. Nothing in the program was simply wrong in the sense of a crashing line of code. The program did what it was built to do. The numbers it received simply did not mean what it assumed they meant. A value is only as meaningful as the unit attached to it, and when two parts of a system disagree about that unit, the disagreement is invisible until the physical outcome arrives.

The diagram shows two sides of the mismatch. The data was written in one system and the program read it as if it were written in the other. The failure lies in the gap between those two assumptions. This is why the case is listed among problems caused by incorrect assumptions as well as by unit mismatches.

A few months after the Orbiter was lost, the Mars Polar Lander probably crashed. Its software was supposed to detect the bump when the spacecraft touched down and then turn off the engines. The source reports that deploying the landing gear might have jarred the spacecraft enough to activate that detection step early.

The program then turned off the engines while the spacecraft was still high above the ground. The source notes that the unpowered craft would have fallen to the surface. The program's logic assumed that a single triggering signal always meant touchdown. That assumption held in the situation the designers expected and failed in the situation that actually occurred.

Read the sequence from top to bottom. The landing gear was not itself the error. The error was that the detection step accepted a signal that did not reliably mean what the software assumed it meant. Each later step faithfully carried out the earlier assumption.

The source points out that the cases get more serious as the work assigned to the computer gets more important. It names three domains where failures directly affect people's health, money, or safety: medical devices, financial systems, and aerospace applications. The two Mars cases belong to aerospace.

Together these cases show that the risk is not limited to a single decade or a single kind of machine. Robust checking matters in every domain where software does work that people depend on.

It is natural to assume that a software failure means a programmer wrote an incorrect program. The Mars Climate Orbiter case shows that this is not always so. The program in that case did what it was built to do. The failure came from a gap between the specification, the assumptions made, and the real physical system.

This is why the source asks you to look at the whole chain from a programmer's decision to a physical outcome. Robustness is a property of the connections between a program, its assumptions, and the world it acts on, not only of the code inside it.