Language Modeling Metrics: Perplexity and Cross Entropy
A judge's score cannot be compared directly across different judges, and judges are themselves systems that get revised.
The Comfortable Belief About Automatic Scores
Many people who build with AI assume that once a model is wrapped in an automatic scoring routine, the output becomes an objective yardstick. The model runs, a judge assigns a number, and the number tells you whether the system improved. That belief is comfortable, and it is the first assumption this article examines. Foundation models break the simple picture in which a test has one right answer and a score can be read straight off the result.
What do you think happens?
If you ask a foundation model to summarize a document, can you check its output by comparing it to one expected string?
Reveal answer
No, many different responses can be valid
A foundation model is open-ended: it can write essays, summarize documents, hold conversations, and produce many valid responses to the same request. There is rarely a single correct string to compare against.
Why Open-Ended Outputs Resist Scoring
The source material makes a clear point: the more capable a model becomes, the more serious its possible failures can be, which raises the importance of evaluation. Yet the same capability is what makes evaluation difficult. The difficulty is not that people have stopped caring about measurement. It is that the outputs themselves have changed shape.
| Feature | Traditional ML model | Foundation model |
|---|---|---|
| Typical output | A narrow answer | Essays, summaries, conversations |
| Checking the output | Compare against an expected value, which is straightforward | No single correct string to compare against |
| Valid responses to one request | Usually one expected answer | Many valid responses to the same request |
| Scoring difficulty | Relatively direct | Harder, because there is rarely one reference answer |
How evaluation changes when outputs become open-ended
Because of this difficulty, many teams turn toward human evaluation. The source treats humans in the loop as useful for sanity checks, and in many cases essential. It also points out an imbalance in effort: investment in evaluation still lags behind investment in building models and applications, even though many new evaluation techniques are being developed.
The source notes that the emphasis on evaluation has been noticed outside research papers. In December 2023, OpenAI cofounder Greg Brockman wrote that evals are surprisingly often all you need. A 2023 study by a16z found that 6 out of 70 decision makers evaluated models by word of mouth, a practice sometimes called a vibe check. These remarks show that evaluation is a practical concern for people deploying models, not only for researchers.
Exact Versus Subjective Evaluation
The source groups the approaches to evaluating open-ended responses into three families: functional correctness, similarity scores, and AI as a judge. The first two are exact. The third is subjective. This split is the key to the rest of the chapter, because the two families behave differently when you try to trust their numbers.
The distinction matters for trust. An exact approach gives a number whose meaning is fixed by the check itself. A subjective approach gives a number shaped by a judgment process, and that judgment process is part of what you are measuring. Reading both kinds of number as if they meant the same thing is the root of many evaluation errors.
Where AI Judges Fall Short
AI as a judge is attractive because it can assess open-ended responses that exact checks cannot handle. A judge model reads a prompt and a candidate output, then produces a rating. The source is careful about the weaknesses of this process. Because the judgment is subjective, the score is shaped by the judge model and by the context in which it is used. Scores have to be interpreted against the specific judge that produced them, not read as universal measurements.
The source's conclusion is balanced. AI judges are promising, but they should be supplemented with exact evaluation, human evaluation, or both. The judge is useful as one instrument among several, and the design question becomes how to combine them.
Ranking Models Without Absolute Scores
There are two broad ways to compare several models. You can evaluate each model independently and then rank them by their scores. Or you can rank them using comparative signals, asking a direct question about a pair: which of the two models is better? The second approach is comparative evaluation. The source notes that it is common in sports, especially chess, and that it is gaining traction in AI evaluation.
Comparative evaluation depends on preference signals, and the post-training alignment process depends on them too. These signals are expensive to collect. That cost motivated the development of preference models, which are specialized AI judges that predict which response users prefer. Preference models are used both in comparative evaluation and in post-training alignment.
Mistakes When Reading Evaluation Numbers
Common mistake Treating an automatic score as an objective yardstick by default
What it looks likeA judge assigns a number to each output, and the team assumes the number is universal.
Why it failsThe source says a subjective score is shaped by the judge model and the context in which it is used.
Do this insteadInterpret the score against the specific judge that produced it and check it with exact or human evaluation.
Common mistake Using a single exact string to check open-ended output
What it looks likeComparing a summary of a document to one reference summary word for word.
Why it failsFoundation models can produce many valid responses to the same request, so rarely is there one correct string.
Do this insteadUse exact checks only where they genuinely fit, and use subjective or comparative methods for open-ended outputs.
Common mistake Using a judge score as a fixed benchmark over time
What it looks likeTracking an application's quality across months with the same judge, assuming its ratings never change.
Why it failsJudges are systems that get revised, so their judgments can shift.
Do this insteadTreat judge-based trends with caution and supplement them with exact or human evaluation.
Common mistake Assuming a comparative ranking is an absolute score
What it looks likeReading a pairwise-based ranking as if each model had a fixed quality number.
Why it failsComparative evaluation ranks models by pairwise signals, not by absolute scores.
Do this insteadRead the ranking as relative standing among the models that were compared.
Applying the Evaluation Split
A team builds a chatbot that answers customer questions in free-form language. They want to know whether a new version is better than the old one. Using the exact and subjective split from this article, name one approach you would consider from each group, state which group is subjective, and explain what supplementing step you would add before trusting a judge's score.
Hints
- Recall which evaluation families the source calls exact and which it calls subjective.
- Think about whether a judge's score could shift if the judge model is revised.
- Consider whether a pairwise question between the old and new version could help.
Key Takeaways
- Foundation models are open-ended and powerful, so they have many valid responses and rarely a single correct string to compare against, which makes them harder to evaluate than traditional models.
- Functional correctness and similarity scores are exact approaches, while AI as a judge is subjective and depends heavily on the judge used.
- AI judge scores may not be comparable across judges and can shift as judges are revised, so they should be supplemented with exact or human evaluation.
- Comparative evaluation ranks models through pairwise questions, and preference models are specialized AI judges that predict user preference, though preference signals are expensive to collect.
- This article does not define perplexity or cross entropy; the source flags these language modeling metrics as frequently confusing and treats them as a separate topic that needs careful interpretation.