Overfitting in GPT Models

After a GPT model has been built and trained, the natural next question is whether it actually works on text it has not learned from directly. A single loss value computed on one piece of text does not answer that question. What matters is how the loss behaves across two different collections of text: the data the model was trained on, and a held-out validation collection that it has not been trained on. Comparing those two numbers is the core of evaluating model performance in this setting.

This article follows the evaluation in the order a practitioner would run it. You will first see how the two losses are calculated, then how to read them, then what a close match between them does and does not prove. The final part covers the fine-tuning and input-format experiments that were reported alongside these measurements, so you can see how changes to the setup were judged.

The evaluation starts with a single pair of calls. Each call passes a data loader, the GPT model, and the device the computation runs on into a function named calc_loss_loader. One call uses the training loader and stores the result as a training loss. The other uses the validation loader and stores the result as a validation loss.

Before reading any output, it helps to notice what is the same between the two calls and what is different. The model and the device are identical. Only the data source changes. Because the model is shared, any difference between the two resulting numbers comes from the data each loader supplies, not from a different model or a different loss function.

The diagram shows the structure to keep in mind. Two separate loaders feed the same model, and each loader produces its own loss. When a result looks unexpected, this structure tells you where to look first: the data entering each branch, or the model that processes both.

The reported losses were computed for the 124-million parameter version of the GPT model. The validation loss is slightly lower than the training loss, and both values are close to each other. That closeness is the observation the rest of the evaluation is built around.

Why does this matter? A large gap in the usual direction, where training loss is much lower than validation loss, is the classic signal that a model has memorized its training data rather than learned patterns that carry over to new text. When the two losses are in the same ballpark, that specific warning sign is absent. Losses that are close together are consistent with no overfitting.

A validation loss slightly lower than the training loss can look puzzling, because it seems to suggest the model is better on unseen text. The source does not read it that way. In a small dataset, a slightly lower validation loss is attributed to random noise rather than to better performance. The practical lesson is to avoid treating a tiny difference in either direction as a finding about the model.

Similar losses can arise for two different reasons, and the difference depends on whether the evaluation text was part of the original pretraining data. The source uses a book called The Verdict as the evaluation text. The two possible cases lead to very different conclusions, so it helps to read them side by side.

In the first case, the text was not part of the pretraining data. The model is then not explicitly overfitting to the training portion, and it performs similarly on the training and validation portions. Here a close match is meaningful evidence against overfitting, because neither portion was seen during pretraining.

In the second case, the text was part of the pretraining data. The validation portion would then also have been seen during pretraining, so the evaluation cannot settle whether the model is overfitting. The source notes that measuring the degree of overfitting would require a new dataset created after pretraining was finished, so that it could not have been included.

Beyond the loss comparison, the source reports how several changes to fine-tuning affected test accuracy. The baseline in the main chapter, where only the final transformer block was fine-tuned, reached 95.67% test accuracy. Every comparison below is measured against that number, so each result tells you whether a change helped or hurt.

The input format is also part of what gets evaluated. The source shows the Phi-3 prompt format, which wraps a user instruction and an assistant response in tags. For the example used, the instruction asks for the correct spelling of a word, and the assistant line gives the correct spelling. Changing the format changes the text the model sees, so it is an experimental variable like any other. The source reports that fine-tuning with the Phi-3 template is about 17% faster, because it produces shorter model inputs. Its reported score is close to 50, which the source describes as in the same ballpark as the score from the Alpaca-style prompts. The speed gain and the similar score are both reported results; the source does not present the format as a clear accuracy improvement.

The second input change masks out the instruction tokens in the target list so that the loss is computed only on the response. The source states that a model fine-tuned this way performs slightly worse, by approximately 4 points, when evaluated with the Ollama Llama 3 method from the chapter. The source ties this to observations in a published paper on loss over instructions. This is a useful reminder that a change which seems principled can still lower a measured score, so each change has to be checked against the same evaluation.

The core of evaluating a GPT model is comparing its loss on training text with its loss on held-out validation text, both computed by the same function and model. The reading of that comparison depends on what the model has already seen. Each reported change to the setup has to be judged against the same baseline before anyone credits it with a result.