Evaluation Data and Guidelines

Imagine you need training examples for finetuning a model. Each example has to be labeled according to what a good response looks like. Attaching a label to an example takes only a few seconds, but deciding which label is correct takes much more thought. This article is about that gap between the physical act of labeling and the standard that gives each label its meaning.

It is natural to think of annotation as a mechanical task: look at an example, choose a label, move on. That picture is incomplete. Annotation is hard not only because of the annotation process itself, but also because creating clear annotation guidelines is complex. The effort of labeling is real, but it is rarely the main source of difficulty.

The difficulty sits in the rules that decide what counts as a correct label. If two people look at the same example and the rules are unclear about where a response stops being acceptable and starts being good, they will reasonably disagree. Their labeling action may be identical in speed and form, yet their labels will differ because they are reading the standard differently or the standard does not settle the question.

Two ideas are easy to blur together. The annotation process is the work of applying labels to examples. The annotation guideline is the standard that tells you what a correct label is. Keeping these apart makes it clear where a problem lies when labels turn out unreliable: in the act of labeling, or in the standard being applied.

A guideline that is usable has to answer at least two questions. It must state what a good response looks like and what makes it good. It must also state where the score boundaries fall, meaning the exact point at which one score ends and the next begins. Without both, a labeler is left to fill the gaps with personal judgment.

Guidelines are needed whether humans or AI systems do the labeling. Relying on a model to find the right answers alone is risky, because the model still has to be told what a correct answer means. The same written standard can feed a human labeler and an AI labeler, and in both cases the standard is what makes the labels checkable.

What differs between the two paths is who applies the standard and how quickly. A human brings interpretation and can misread a vague rule. An AI labeler applies whatever it is given consistently and fast, but it cannot resolve a gap in the rule by knowing better. In both paths the quality of the labels depends on the quality of the guideline.

The annotation guidelines for finetuning data are the same as the guidelines for evaluation data. This overlap is useful to know: the care you put into writing a clear standard for evaluation pays off again when you label training data, and the reverse is also true. Investing well in evaluation guidelines and evaluation data therefore serves more than one purpose.

Evaluation examples can also be augmented or used as seed examples to synthesize new data. Data augmentation and data synthesis are both processes commonly used to generate data programmatically. Both start from the same guidelines, so a weak guideline weakens the generated data as well as the labeled data.

Follow what happens to a guideline after it is written. It does not label anything by itself. A labeler, human or AI, reads an example, applies the standard, and produces a label. The labels then have to be checked against the same written standard, because that is the only fixed reference point available.