Training a Large Language Model

Training a language model means repeatedly updating its weights and watching the loss. When the loss goes down, that is encouraging, but it does not by itself answer the question that matters most: does the training actually improve the model? A falling number on one dataset can hide a problem, and it cannot tell you how the model behaves on text it has not learned from.

The course handles this with a separate measurement step. After each model update, the training process measures the model's loss on both the training data and held-out validation data. In this course that step is carried out by a function named evaluate_model. It takes the model, a training data loader, a validation data loader, the device the computation runs on, and an eval_iter value that controls how many batches are used. It returns two numbers: a training loss and a validation loss. Progress is judged by these measurements rather than assumed.

The rest of this article follows the function from the outside in. First you will see where it fits in the larger course path, then what it does to the model's behavior, then the exact order in which it runs its steps. Keep one question in mind throughout: why does each line exist?

Imagine you want a program that can write fluent sentences. You could try to write grammar and word-choice rules by hand, but language is too varied for that to work. A different approach is to let a model learn patterns from a large body of text and then use what it learned to generate new text. The process of building that learned starting point is called pretraining. It is the stage where a model is first trained on text before anyone asks it to do a specific job.

The course divides this work into two stages. The first stage prepares the ingredients: the data preprocessing steps and the attention mechanism at the core of modern LLMs. The second stage puts those ingredients together, showing how to code and pretrain a GPT-like model that can generate new text, and then how to evaluate the result. Keeping these stages in mind helps you see why each topic exists.

The diagram below lays out the path from raw data to a judged model. Read it from left to right. The first two steps belong to stage 1. Pretraining belongs to stage 2. The branch for loading open model weights sits beside pretraining because it is an alternative way to reach a trained model without running the full training yourself. Both routes end at evaluation, because a model must be measured whichever way it was obtained.

Data preprocessing is a crucial step in pretraining an LLM. Raw text is rarely in a shape a model can learn from directly, so it has to be prepared first. Preprocessing is the part of the workflow that turns collected text into training material. Because every later step depends on this preparation, it is worth understanding before you move on to the model itself.

The attention mechanism is the second ingredient from stage 1. The source describes it as being at the heart of every LLM, and it is the component the course asks you to code yourself. Think of attention as the part of the model that lets each position in a sequence take other positions into account when it builds its representation. Stage 1 builds this mechanism in isolation, so that stage 2 can use it as a working part rather than a mystery.

Put together, the two ingredients explain the order of the stages. Text must be preprocessed before a model can learn from it, and the model's design must include attention before pretraining can use that design. Only when both are in place does the training run in stage 2 make sense.

The pairing at the heart of evaluate_model is simple. The same measurement is taken on two different datasets, so the two resulting losses can be compared. The training loss shows how the model performs on data it has learned from. The validation loss shows how it performs on data kept aside for checking. Comparing these two numbers after each model update is what allows the training process to be judged.

Read over several evaluations, the two losses tell a story. If both fall, the model is generalizing along with the training data. If the training loss keeps falling while the validation loss stops improving, the model is fitting the data it has seen without carrying that improvement over to held-out text. The source does not give a fixed threshold for this judgment, so treat the pattern as the thing to watch, not a rule with a number attached.

Why measure after each update rather than only at the end? Because a training run is a sequence of changes to the weights. Each evaluation is a checkpoint that asks whether the latest change helped. Without these checkpoints, progress is assumed rather than observed.

Keep the earlier question in mind: the function does more than compute numbers. Before any loss is calculated, it changes how the model behaves for the duration of the calculation. Two effects are involved, and they are separate ideas. The comparison diagram below sets training mode beside evaluation mode so you can see which behavior changes and which stays the same.

What stays the same is the model itself. Evaluation mode does not change the weights, and it does not change the data being measured. It changes only the two behaviors described below, and only for the length of the evaluation.

The order of the function is significant, and it is the part most worth memorizing. The mode is set before any loss is computed, so both losses are measured under the same conditions. The mode is restored only after both losses have been computed, so the switch back to training happens at the end of the function, not in the middle of the measurement. The flowchart below traces the steps in the order they run.

Pretraining an LLM from scratch is a significant undertaking. The source states that it demands thousands to millions of dollars in computing costs for GPT-like models. That range is wide, and the bottom and top ends differ enormously, so the practical lesson is that full-scale pretraining is a major commitment rather than a routine exercise.

This cost is the reason the course narrows its goal. Instead of pretraining at full scale, stage 2 focuses on implementing training for educational purposes with a small dataset. Loading openly available model weights is the alternative route. Both routes still require evaluating the resulting model, which is covered in the next section.

The third topic in stage 2 is evaluating LLMs. The source describes this as essential for developing capable NLP systems. A trained model is a set of learned values, and those values alone do not tell you whether the model produces good text. Evaluation is the step that measures what the model does, so that you can compare versions and decide what to change.

Evaluation applies to both routes. A model you pretrain on a small dataset needs to be evaluated, and so does a model whose weights you loaded from an open source. Because the weights in the second case came from elsewhere, you cannot assume they behave as you expect in your own setting. Measuring is how you find out.