Eval Discipline: Matching Checks to Case Difficulty

A project often begins with a clear picture of what to build. The harder question comes next: how will you know that what you built actually works? In AI engineering, the answer is an artifact called the eval set. This article argues that the eval set is not a formality added at the end of a project. It is the thing that decides whether the project succeeded.

The gap between a persuasive demo and a failed eval run is the reason eval discipline exists. Before this article moves on, keep one question in mind: what does the system have to be measured against, and who decided what that measure is?

In traditional software development, the specification is a document. It describes what the system is supposed to do, what inputs it takes, what outputs it produces, and what edge cases it has to handle. Engineers read the specification and write code that satisfies it. Tests then check that the code matches the specification. In this arrangement, the specification is the source of truth.

Trace the flow step by step. The specification is written first, in prose. The code is written to satisfy that prose. The tests check the code against the prose. Whatever defines correctness sits in a written description, and everything downstream is judged against it.

In AI engineering, the specification is the evaluation set. The model plays the role of the implementation, and the implementation is judged against the eval set. The eval set is the source of truth. This may sound like a small change, but it inverts the engineering process. The thing that defines correctness moves from a description written in prose to a collection of concrete examples that the system is measured against.

Read the two columns as a parallel. In the left column, a written description is the source of truth and code is the thing being checked. In the right column, concrete examples are the source of truth and the model is the thing being checked. The change is not cosmetic. Correctness is now defined by examples, so the quality of those examples determines what counts as correct.

Notice the hierarchy. The eval set is a collection, each case is one pair, and the input and expected output are the core of every case. The rubric is not required for every case. It appears when the right judgment is about qualities of a response rather than an exact match.

The eval set can fail in two directions. If it tests for things the customer does not care about, effort goes into the wrong checks. If it omits things the customer does care about, the checks are silent on exactly the behavior that matters most. In either case the eval set is wrong or incomplete, and the consequence is the same: a system that passes a poor eval set will still fail in production.

The downstream pattern is predictable. The system ships, performs unevenly in production, and the team spends months debugging in production what should have been caught during validation. Because the eval set decides whether validation succeeds, and validation decides whether the engagement succeeds, an eval set that misses real cases undermines the whole project. The reverse is also part of the rule: a system that fails the evals on its first run is not yet ready to ship.

Once the eval set exists, the team still has to decide how each case will be checked. The course presents a three-level ladder for eval discipline, and each level matches a different kind of case. Follow a case through the ladder below. Start at the top of the flow with an incoming case and ask the question at each decision point.

Read the flow as a routing rule. The simplest check is tried first, because it is the cheapest and most direct. Only the cases that check cannot settle move on. Each level is defined by the kind of case it handles, so the ladder is less a sequence of steps than a match between case difficulty and check depth.