Reading Traces for AI Debugging

Imagine two teams that ship the same AI assistant. Both watch a dashboard full of averages. Both see the numbers drift slightly downward in the third week after launch. The first team responds by tweaking settings and hoping the averages recover. The second team opens a handful of individual conversations, reads what the user actually typed and what the assistant actually answered, and discovers that one kind of request is being handled badly again and again. The second team has begun working through the per-trace loop. This article explains how that work is organized, and why it is the process that turns a fragile launch into a product people keep using.

The discipline asks for something that feels slow. You read the input the user supplied and the output the product produced, in their exact wording. Summary numbers compress many exchanges into one figure, and that compression removes the specific words where a failure lives. Reading the raw exchange keeps those words visible, so you can see what the user asked and what the product did with it.

It helps to see the loop as a sequence in which each stage produces something the next stage uses. Reading sampled traces produces a hypothesis. The hypothesis is encoded as an eval case. The eval case is added to the standing eval suite. The suite is then used to grade a fix. The fix is not the end of the chain: its grade comes from the suite, which already holds everything the team has learned so far. The diagram shows the loop as a repeating cycle, so you can follow what each pass hands to the next.

A single trace rarely tells you enough by itself. The loop asks you to read enough traces in sequence to form a working hypothesis about what is going wrong. The hypothesis is therefore a pattern you notice across several exchanges, not a conclusion drawn from one unusual reply. An odd answer in one exchange may be a one-off. The same kind of request handled badly in several separate exchanges is the signal the loop is looking for.

A hypothesis is useful only if it can be checked again later. The loop therefore encodes it as an eval case: a test built from the hypothesis that captures the kind of input that exposed the problem and the behavior a good answer should show. Writing the case is what converts an observation made while reading into something the team can run repeatedly.

Continuing the refund example, the eval case would capture a refund-status request as its input and describe the behavior a good answer should show, which is a status answer rather than a general policy answer. The case is built from the pattern, so it tests the same kind of request every time the suite runs.

The eval case then becomes part of the standing eval suite. The word standing matters. The suite is not rebuilt for each change; it accumulates. Each round of reading leaves behind a test, so the suite records everything the team has learned about where the product fails.

When a fix is proposed, it is graded against the suite. The fix must handle the new case and must also keep passing the cases the team wrote earlier. A fix that repairs the refund-status problem but breaks an earlier case does not pass the grade. The flowchart traces that path, including the two outcomes a proposed fix can reach.

The source describes the loop's shape as unglamorous, and it is also what produces the week-over-week improvement curves that separate a deployment customers keep using from one they quietly turn off about six weeks after launch. Those curves are not a single event. Each cycle of reading traces, forming a hypothesis, encoding it, and grading a fix adds to what the team knows and what the suite checks, so improvement accumulates over time.

Because every cycle leaves an eval case behind, the suite grows alongside the product. Later fixes are graded against everything already learned. That is why the loop is best understood as the mechanism behind sustained improvement rather than as a one-off review. Repeating the loop is the work that keeps the curve rising.