Interpreting Training Curves

When a model is fine-tuned, its loss is recorded again and again during training, and also on a held-out validation set. A single final number cannot tell you how that process unfolded. A plot can. It shows whether the loss fell quickly, leveled off, or moved in a direction you did not expect, and it lets you compare the behavior on the training data with the behavior on data the model never trained on.

In this article you will look at a small plotting routine and then work backward through each part of it. The goal is to be able to look at the resulting picture and say, with specific reasons, whether the run went well and, if it did not, which part of the pipeline you would inspect.

A loss curve is a record of how far the model's predictions are from the correct answers at each point in training. Plotting it against epochs puts the training history on a horizontal scale that counts full passes through the training data. The training loss is measured on data the model is learning from. The validation loss is measured on data held back from training, so it indicates how the model behaves on material it has not been fitted to.

Both lines are drawn from their own list of values and share the epoch axis. Neither line is computed from the other. That independence is what makes the comparison meaningful: if the two lines end up behaving alike, that similarity is evidence rather than something the plotting code forced.

In the example this routine is built for, both lines fall sharply during the first epoch. After that, they flatten out gradually and settle toward the fifth epoch. Two readings follow from that shape. The first is that the model is making real learning progress, since the loss drops quickly at the start. The second is that the model learned from the training data and carries that learning over to the unseen validation data, because the validation line follows the training line rather than separating from it.

The two columns of a comparison have the same shape, and that similarity is the signal. When the training and validation curves move together, the model is generalizing. When one line keeps falling while the other does not follow, the picture no longer supports that conclusion, and you should read the two lines together rather than in isolation.

The routine creates one figure with one axes object, plots both loss lines on it, labels the horizontal axis as epochs and the vertical axis with the loss name, and adds a legend so the two lines can be told apart. Those are the familiar parts of any line plot. The distinctive part comes next: the routine needs a second horizontal scale for examples seen, and that scale is drawn on a separate axes object that shares the same vertical range.

The plotting code can be read as a sequence of stages. Each stage has a specific job, and each produces something the next stage depends on. Following the stages in order makes it possible to see where a problem would enter the picture. The stages redraw only what they are given, so a line that diverges from expectations can come from the lists passed in, the order of arguments, or the epoch positions, before the model itself is blamed.

The layout adjustment comes after both axes exist, which is why it can make room for the top label as well as the bottom one. The save and show steps come last, so whatever is on the figure at that point is what gets written to the file and displayed. A mismatch between the expected and actual curves therefore has a short list of places to check: the two input lists, the argument order, and the epoch positions.