Evaluating Open-Ended Model Outputs

Most people who have trained a small predictive model learn to judge it with a simple routine. You prepare inputs whose correct answers are known, run the model on them, and count how often its output matches the expected answer. For a model that chooses from a fixed menu of possible results, this routine works well. Foundation models break it in several ways at once. The sections that follow trace three of those breaks, then look at a fourth change in what evaluation is supposed to accomplish. Along the way you will see why a score that looked meaningful for one generation of models can stop telling you much about the next.

Open-ended tasks change the structure of the check described above. Exact-match comparison asks whether the output is one of the expected answers. That question has a clear meaning when the expected answers are a short list. It has no clear meaning when a single request can be answered correctly in countless different wordings.

The two columns of the diagram show the core problem. In the exact-match column, the key is complete enough that a missing match means the answer is wrong. In the open-ended column, the key can only cover some of the valid responses. An answer key that covers only some valid responses will mark good answers as wrong, and it gives no information about the responses it never listed.

Exact match is incomplete for open-ended tasks in two directions at once. It produces false failures, because a correct response that happens to be absent from the key is scored as wrong. It also produces silence, because a response outside the key tells the evaluator nothing about quality. A high match rate against a partial key therefore does not mean the model is good at the task. It only means the model agreed with the listed cases.

A black box model is one whose internal details are hidden from the evaluator. The architecture, the training data, and the training process are not visible. These details matter because each can reveal where the model is likely to be strong and where it is likely to fail. When they are hidden, the evaluator loses the most direct explanations for the model's behavior.

The diagram shows the evaluator's position. Inputs go in and outputs come out, and those are the observable facts. Everything inside the model box stays out of view. When the evaluator wants to explain a failure, the explanation has to be inferred from the pattern of outputs rather than read directly from the model's design or training records. Testing therefore becomes an exercise in watching behavior from outside.

A benchmark is saturated for a model once the model achieves the perfect score. Foundation models saturate benchmarks quickly, and this forces repeated replacement. A saturated benchmark can no longer separate a strong model from a stronger one, because both reach the ceiling. The score still reports a number, but that number has stopped carrying information about differences between models.

The pattern in the table is consistent. Each older benchmark was once useful for measuring models, then either saturated or was overtaken by stronger models, and a successor was introduced. The source notes that GLUE became saturated within about a year, while MMLU was a strong benchmark that many early foundation models relied on before being largely replaced in 2024. The lesson is that a benchmark's usefulness is temporary and depends on how far models have already progressed.

Evaluating a general-purpose model has a second role beyond checking known abilities. The source describes evaluation as both measuring known tasks and discovering new tasks, which may include tasks beyond human capability. An evaluator is not only checking a list of expected abilities. The evaluator is also trying to find abilities nobody wrote down in advance and boundaries nobody has yet mapped.

The cycle in the diagram explains why a fixed benchmark cannot be the whole evaluation effort for a general-purpose model. A known benchmark measures listed abilities. Once it saturates, it stops being informative, so new tasks must be found and turned into new tests. Those tests then saturate in their turn. The source also reports that the community's response has been substantial: the number of published papers on LLM evaluation grew exponentially during the first half of 2023, rising from about 2 papers per month to almost 35 papers per month.