Monitoring Production AI Systems

Imagine an application built from many interacting components, including language models, retrieval steps, tool calls, and a web front end. A change goes out on a Tuesday afternoon. Nothing crashes, no alarm fires, and the team goes home. Two days later a customer reports that answers have quietly become wrong for a subset of questions.

The central question of this article is not only how the failure happened, but how long it went unseen and how long it took the team to repair it once they knew. Those two durations are what the course's metrics are built to measure, and the rest of the article shows how to read them.

The source argues that visibility into a system should be designed in from the start rather than bolted on afterward. The reasoning is that the more complex a product is, the more costly it becomes to be blind to what it is doing. A single component can often be understood by reading its behavior directly. Once many components interact, a wrong answer at the front end may originate several steps away, in a retrieval step or a tool call, and nothing about the final output points to where.

This is why observability matters more as architecture grows. Each added component and each added dependency between components increases the number of places a fault can hide, while the outputs the user sees stay the same shape. The practical job of observability is to make sure the system exposes enough information that its internal state can still be understood from what it produces.

To make the metrics concrete, attach them to a single failure as it unfolds. The sequence has four moments in order. First, a change is deployed. Some time later the failure actually begins, meaning the system starts behaving incorrectly. At some later point, someone or something notices the problem. Finally, the problem is resolved and the system behaves correctly again.

The first gap runs from the moment the failure begins to the moment it is detected. The source's metric MTTD, mean time to detect, is the average of this gap across many incidents. The second gap runs from detection to resolution, and the metric MTTR, mean time to resolve, is the average of that gap. Together they split the failure's lifetime into two consecutive intervals. The gap between deployment and the start of the failure is not one of the two measured intervals.

The two gaps carry different meanings. The failure was already real before anyone knew about it, so a long detection gap means the system spent that period producing wrong output with nobody acting on it. A long resolution gap means the team knew about the problem but the repair took time. A system can have a fast detection gap and a slow resolution gap, or the reverse, and the two numbers point to different weaknesses.

Monitoring and observability are easy to blur together, but they describe different things. Monitoring is the act of tracking a system's information. Observability is the larger process that contains monitoring along with instrumenting the system and debugging it. In the source's convention, observability covers the whole cycle of instrumenting, tracking, and debugging, not just the act of watching.

The sharpest difference lies in what each approach assumes about the link between a system's outputs and its internal state. Traditional monitoring makes no such assumption. You watch external outputs in order to notice that something has gone wrong inside, but nothing guarantees that those outputs will reveal what the internal fault actually is. Observability asks more of the system. It assumes that internal state can be reasoned about from outputs, so a well-instrumented system should let you pinpoint a cause from its logs and metrics alone.

Monitoring still has real value on its own. The source gives its purpose as helping to mitigate risks and to discover opportunities. The risks named include application failures, security attacks, and drifts in behavior. Monitoring can also surface opportunities for improving the application and saving cost, and it keeps the people responsible for a system accountable by making its performance visible.

Monitoring is not meant to operate alone. The source describes evaluation and monitoring as working closely together. Metrics used during evaluation should translate well into metrics used during monitoring, so that a model which performs well in testing also performs well in production. The loop runs the other way as well: issues that monitoring detects in production should be fed back into the evaluation pipeline, so that the next round of testing can catch them before deployment.

Each pass through this loop leaves the system in a different state than before. The issue that monitoring surfaced now belongs to the set of cases the evaluation pipeline checks, so the next round of testing has a chance to catch a repeat before it reaches users. The loop is what turns a production incident into a test that protects future releases.

The source names three metrics drawn from the DevOps community. The first two measure the gaps in the timeline. The third measures how often changes cause problems in the first place. Each is an average or a percentage over many events, so a single incident says very little on its own. For a set of n incidents, let d_{i} be the detection gap of incident i and r_{i} its resolution gap. Then:

CFR, the change failure rate, is the percentage of changes that need fixes or rollbacks. Over a set of deployments it is the number of changes that needed a fix or rollback divided by the total number of changes, expressed as a percentage.

Read together, the three numbers give a fuller picture than any one of them. MTTD and MTTR describe how quickly a system's problems become visible and how quickly they are repaired. CFR describes how often changes cause those problems at all. A team that watches only one of these numbers can be misled about where its weakness really lies.