Designing Failure Mode Panels

Imagine two status reports arriving from the same vendor each month. Both report the same uptime, the same throughput, and the same row of green indicators. The first report stops there and never mentions a problem. The second report also includes a short entry describing an error that reached customers, what was changed to stop it, and how the team will check for it next time. The question this article works through is which report a careful reader should believe.

The answer is not obvious if you assume that metrics are what persuade people. A dashboard is a set of measurements, and it is easy to assume that those measurements carry the persuasive weight. The source material for this lesson argues that something else often does the persuading: a willingness to show where things went wrong. This article traces how that willingness changes a reader's trust, step by step.

Follow the reader's trust through the two reports. With the first report, the reader sees a clean summary and no failures. The metrics are as good as the reader can check, but nothing in the report shows how the vendor behaves when something goes wrong. The reader has no evidence about that behavior, so the reader has to decide how much to rely on the summary without it.

With the second report, the reader now has evidence about behavior. The report names an error that reached customers, which shows the vendor is willing to say so. It names the change that stopped the error, which shows the vendor acted. It names the check for next time, which shows the vendor is still watching. Each of these facts adds to the reader's picture, and the reader's trust in the other numbers in the report moves accordingly.

The source describes a dashboard built to track failures and metrics for a team working on an AI system used in clinical documentation. The most valued part of that dashboard was a panel devoted to failure modes. The executive sponsor, a chief marketing officer, valued that panel most and said so directly during the engagement.

The reason this mattered is a history of distrust. The same executive had said she had stopped reading vendor status reports years earlier, because she believed vendors concealed bad news. Against that background, the dashboard became the first artifact she trusted. The source treats this as the central purpose of the dashboard: trust earned through visible honesty about failure, rather than trust demanded through a clean-looking summary.

A specific, concrete description of a failure is more credible than a vague one. A panel that never shows problems can invite doubt, because a reader who has seen any real system behave imperfectly may reasonably wonder what is being left out. Honesty about failure therefore does not weaken the dashboard. It is part of how the dashboard earns belief in everything else it shows.

When the team first displayed a real failure on the dashboard, the report recorded three things, in a fixed order. Each part answers a different question a skeptical reader would ask. The diagram below shows how the three parts hand off to one another.

A report that contains only the first part tells the reader there was a problem. A report with all three tells the reader that the problem was understood, repaired, and placed under ongoing watch. Each step adds a reason the reader can rely on, and removing any step leaves a question unanswered.

The safeguard matters most after the fix is in place. Consider what happens if the same failure returns. The diagram below traces the path a failure takes once the safeguard exists, compared with the path it would take without one.

Notice how the safeguard changes the path. Without an evaluation case, a returning failure has no reliable detector, so its return depends on someone happening to notice it. With the case in place, the same class of failure is detected by design. The reader of the next report can see that loop, and the continuing check is part of what the reader is trusting.

A helpful way to think about a dashboard is to separate two layers. The first layer is the content: the metrics, the counts, and the charts. The second layer is the stance the content takes toward the reader, which shows in what gets displayed and what gets left out. The source frames the metrics as the medium through which the team communicates, and the honesty as the thing actually being communicated.

Once you see the two layers, the design question changes. Instead of asking only whether a metric is accurate, you also ask whether the dashboard gives a reader reason to believe that accurate metrics would still be shown if they were unflattering. A dashboard can have correct numbers and still fail this second test if it never displays a problem.

The same content can build trust or erode it, depending on what the report does with its failures. The diagram below contrasts a panel that erodes trust with one that builds it, using the same underlying metrics.