Designing Evaluation Rubrics for AI Outputs
Criteria drift is the change in evaluation criteria that occurs as a rubric is used to grade real outputs and surfaces unanticipated cases.
Why a Careful Rubric Still Changes
Many people who build evaluation systems for AI outputs begin with a plan that sounds entirely reasonable. They sit down, think carefully about what a good answer looks like, write a complete rubric with clear criteria and scoring levels, and only then begin grading. The rubric feels like a specification, and a specification written with care seems like it should hold up once real outputs arrive.
This article examines that plan and asks what happens to it once real outputs arrive. The short answer is that the rubric you wrote first will almost certainly change, and the change is not a sign that you did something wrong. Understanding why it changes is the first step toward building evaluation criteria that actually work.
What do you think happens?
A team writes a careful rubric with five criteria and then grades the first fifty real model outputs using it. What is most likely to happen to the rubric?
Reveal answer
It changes, because the criteria become concrete only as the outputs are graded.
The source reports that practitioners cannot fully define evaluation criteria up front. The first version of a rubric is expected to be incomplete, and the grading itself reveals the cases the rubric did not anticipate.
The Belief in a Finished Rubric
The misconception behind the careful-plan approach is the belief that the criteria for a good output can be fully specified before anyone has looked at a single real output. Under this belief, grading is a mechanical step performed after the real design work is finished. If the rubric turns out to be incomplete, the designer is treated as having failed to think hard enough.
The source identifies this belief as the misconception to avoid. Its appeal is understandable, since a designer who thinks hard before grading is doing something sensible. The problem is the assumption that thinking alone can surface every case the rubric will need to handle. Real outputs contain cases that no one imagined while drafting, so the belief sets an expectation that cannot be met.
What Criteria Drift Means
Criteria drift is the change in evaluation criteria that occurs as a rubric is used to grade real outputs and surfaces unanticipated cases.
Shankar's empirical finding, reported in her UIST 2024 paper Who Validates the Validators?, is that practitioners cannot fully define evaluation criteria up front. The criteria become concrete only as the practitioner applies them. In practice, the criteria crystallize through grading.
It helps to think of drift as a chain of consequences. You apply the rubric to outputs. Some outputs fall into cases the rubric never considered. Those cases force a change to the rubric. Because the rubric is the definition of the criteria, changing the rubric also changes the criteria. The drift is the cumulative movement of the criteria along that chain.
Tracing One Rubric Through Real Outputs
A summary rubric meets a partly correct output
A team drafts a rubric for grading AI-written summaries of a short report. The first criterion reads: the summary is accurate. The team grades a batch of outputs and finds a summary that gets the main finding right but misstates one secondary figure. The team must decide how to score it.
- 1Apply the initial rubric
The grader checks the summary against the single accuracy criterion. The summary contains one error, so the grader must choose between a pass and a fail, and the rubric offers no middle option.
- 2Notice the unanticipated case
The output does not fit either side of the original binary judgment. It is neither fully accurate nor wrong. This is a case the rubric never considered, which is exactly the kind of case that surfaces through grading.
- 3Revise the criterion
The team adds a scoring level for partly correct summaries and defines what separates it from a full pass. The criterion now means something different from what it meant before grading began.
- 4Recognize the drift
The criterion has moved from accuracy as a yes or no judgment to accuracy with graded levels. That movement is criteria drift. It was produced by real outputs, not by an error in the original thinking.
The rubric changed because grading exposed a case the first version could not handle. The change moved the criterion itself, which is what criteria drift describes.
The Draft, Grade, Revise Cycle
Because criteria drift is expected, the source recommends a working method instead of a perfect first attempt. The method is iterative, and its steps are simple to state. Each pass produces a rubric that is tested against examples, and each test yields the material for the next revision.
- 1Draft a rubric with the criteria and scoring levels you can currently state.
- 2Grade example outputs with the customer's expert.
- 3Record the disagreements between the rubric and the expert's judgments.
- 4Revise the rubric to address the recorded disagreements.
- 5Repeat until the rubric stabilizes.
Mistakes Teams Make With Rubrics
Common mistake Treating the first rubric as the final specification
What it looks likeA team grades fifty outputs with its opening rubric and then declares the grading finished because the rubric was written carefully.
Why it failsThe source states that the first version of a rubric is always wrong, and that criteria become concrete only through grading.
Do this insteadTreat the first rubric as a draft to be tested, and expect it to be revised after the first round of grading.
Common mistake Reading a revision as a sign of failure
What it looks likeA designer sees a criterion split into new levels after grading and concludes the original thinking was careless.
Why it failsThe change is criteria drift, which the source describes as expected and not a sign that something was done wrong.
Do this insteadRecord the revision as evidence that real outputs surfaced a case the draft did not cover.
Common mistake Stopping after one pass
What it looks likeA team revises the rubric once and begins using it for the full evaluation without grading the examples again.
Why it failsThe cycle is designed to repeat until the rubric stabilizes, and the source expects several passes rather than one.
Do this insteadRepeat the draft, grade, revise cycle until the rubric stops changing in response to new disagreements.
Common mistake Grading without the customer's expert, so disagreements are never recorded against the right judgment
Planning a Rubric Revision Cycle
A team has drafted a rubric with four criteria for grading AI-generated answers to customer questions. In its first grading round, the expert disagrees with the rubric on three of twenty outputs, and two of those disagreements involve the same criterion, tone. Describe what the team should do next, using the draft, grade, revise cycle. Explain which criterion you expect to drift and why.
Hints
- Start by identifying which recorded disagreements feed into the revision step.
- Think about what the repeated disagreements on tone suggest about the criterion's wording.
- Remember that the cycle repeats until the rubric stabilizes, not after one revision.
Key Takeaways
- Criteria drift is the change in evaluation criteria that occurs as a rubric is used to grade real outputs and surfaces unanticipated cases.
- Practitioners cannot fully define evaluation criteria up front, so the first version of a rubric is expected to be incomplete.
- The belief that a complete rubric can be written before any grading begins is the misconception to avoid, because the criteria become concrete only through grading.
- The recommended method is iterative: draft a rubric, grade examples with the customer's expert, record disagreements, revise, and repeat until the rubric stabilizes, with about three iterations as a guideline.