Evaluating Response Quality of an Instruction-Tuned Model
After fine-tuning a language model on instruction data, the natural first question is whether training actually worked.
Why Check Learning With a Plot
After fine-tuning a language model on instruction data, the natural first question is whether training actually worked. A single number at the end of a run can hide what happened along the way, so the approach described in the source answers this question with a plot instead. A plot of training loss and validation loss over time shows how the model's error changed across the whole run, and it lets you see the overall trend at a glance.
The plot is produced with the same plotting function that was used during pretraining. Before the plot is drawn, the code builds an array of epoch positions that matches the number of recorded training losses. That array, together with the tokens-seen values and both loss lists, is passed to the plotting function. You then run the call and look at the resulting plot before reading any further. Pay attention to where your expectations and the actual lines differ, because that gap is where most of the useful learning happens.
What Each Curve Tracks
The plot contains two lines that are read together. The training loss curve is a plot of the training loss over time, and it reflects the model's learning progress as it works through the training data. The validation loss curve is the same kind of plot, but it is computed on the validation set. The difference between the two lines is the difference between the data they are measured on, and that difference is what gives the validation line its meaning.
The validation line matters because the validation data is held apart from training. The source states that the validation curve follows a similar pattern to the training loss, with a rapid decrease during the initial phase. A validation curve that falls alongside the training curve is therefore evidence that the improvement is not limited to the examples the model has already seen. The two lines are read as a pair, and the source's conclusion rests on the behavior they share.
The Sharp Drop and the Gentle Slope
The typical shape of both curves has two distinct phases. At the start, the training line drops sharply. The source reads this as the model quickly picking up meaningful patterns and representations from the data. After that early drop, the line keeps falling but more gently. The source reads this slowdown as the model fine-tuning the representations it has already learned.
These are two different kinds of progress, and the shift between them is worth noticing. When you read a plot, locate the moment the line stops dropping steeply and begins to slope. Before that point the model is picking up broad patterns. After it, the model is mostly refining what it already represents.
Where Losses Stabilize
Convergence is the process by which the model's losses stabilize. In the source's reading, the losses continue to decrease as training progresses, but at a slower rate. This slowdown indicates the model is converging to a stable solution. The model has learned meaningful patterns from the data, and further training is refining rather than discovering.
The point where both lines begin to stabilize is the point you are looking for when judging convergence. It is identified by reading the two curves together, because the source's conclusion depends on both lines improving with shared behavior. A line that is still falling steeply has not yet reached that stage, even if it is clearly improving.
Worked Reading of a Run
Reading a loss plot step by step
You have just run the plotting call after a fine-tuning run on instruction data. Describe how you would read the training and validation curves, and decide whether the run looks like it is converging.
- 1Look at the whole run first
Before zooming in, check the overall trend of both lines. Both should move downward over the run, which is the first sign that the model's error is decreasing.
- 2Find the sharp phase
Locate the early region where the training line drops steeply. The source reads this as the model quickly picking up meaningful patterns and representations.
- 3Find the gentle phase
Move right along the training line until the drop becomes gradual. This is the shift from picking up patterns to fine-tuning representations already learned.
- 4Check the validation line against the training line
Confirm that the validation line follows a similar pattern, with a rapid early decrease. Shared behavior is the evidence that improvement is not limited to the examples the model has already seen.
- 5Locate where both lines level off
Identify the point where both losses begin to stabilize. Continued slow decrease that flattens out is consistent with convergence to a stable solution.
- 6State what the plot does and does not show
Conclude that the plot shows learning progress. It does not show whether the model's answers are correct, which must be evaluated separately.
A run is consistent with the expected pattern when both lines drop quickly, then decrease more slowly and level off together. That reading supports convergence, but it does not confirm response quality.
Mistakes When Reading Loss Plots
Common mistake Treating a falling loss curve as proof that the answers are good
Why it failsThe loss curves show whether the model is learning. They do not, by themselves, tell you whether its answers are good. Response quality and correctness must be evaluated separately.
Do this insteadUse the plot as a first check of learning, then evaluate the responses themselves.
Common mistake Reading only the training line
Why it failsThe training line reflects progress on data the model is learning from. Without the validation line, you cannot tell whether the improvement extends to examples the model has not seen.
Do this insteadRead the two curves together and check whether the validation line follows a similar pattern.
Common mistake Deciding the whole run failed when one part of the plot looks off
Why it failsReading a loss plot is a comparison between expected and observed behavior. A single departure does not mean the whole run failed.
Do this insteadFind the exact point where the observed behavior departs from the expected pattern, and use that point to guide the next check.
Common mistake Calling a line converged while it is still dropping steeply
Why it failsConvergence is marked by losses that stabilize as the decrease slows. A steep drop is the sharp early phase, not a stable state.
Do this insteadWait for the slope to flatten and for both lines to level off before reading the run as converging.
Checking Your Reading
What do you think happens?
After a run, the training loss has fallen steadily and both curves have leveled off. Does this prove the model's answers are correct?
Reveal answer
No, the loss curves show learning progress but not answer quality
The source is explicit that loss curves tell you whether the model is learning, not whether its answers are good. Response quality and correctness must be evaluated separately.
Suppose the training line drops sharply in the first part of the run and then keeps falling more gently, while the validation line shows a similar early drop. Describe which part of the run represents picking up broad patterns, which part represents fine-tuning, and where the losses begin to stabilize.
Hints
- The sharp early drop is the model picking up meaningful patterns and representations.
- The gentler continued fall is the model fine-tuning representations it already learned.
- Stabilization is where the losses level off together and the decrease slows toward a stable solution.
Summary of Reading Loss Curves
- 1The training loss curve shows the model's learning progress across the training data, and the validation loss curve shows the same kind of progress on held-out data.
- 2Both curves typically drop rapidly at first, then decrease more slowly as the model converges.
- 3Stabilization of both losses, with the decrease slowing toward a stable solution, suggests the model is converging and further training is mainly refining.
- 4When a curve departs from the expected pattern, find the exact point of departure to guide the next check rather than judging the whole run.
- 5Loss curves show whether the model is learning. Response quality and correctness must be evaluated separately.
At a glance
- The training and validation loss curves together show how learning progressed across the run.
- Both curves usually drop rapidly at first, then decrease more slowly as the model converges.
- Losses that stabilize as the decrease slows suggest the model has converged to a stable solution and that further training is refining rather than discovering.
- A departure from the expected pattern should be traced to the exact point where it begins, not used to judge the whole run.
- Loss curves show learning progress only; response quality and correctness must be evaluated separately.