Training and Validation Set Losses

Training a large language model produces a stream of numbers, and one of the most important is the loss. A single loss value on its own does not tell you very much. Is the model learning something useful, or is it only getting better at the particular text it has already seen? This article focuses on one practical answer to that question: computing the loss on two different sets of data, the training set and the validation set, and reading the two results together.

The source states the purpose directly: computing the training and validation set losses is a way to assess the quality of LLM-generated text during training. As you read, keep one question in mind: what does each of the two losses tell you that the other one cannot?

The script under study is small, but almost every line answers a practical question. Which hardware should run the computation? Should the program keep track of gradients? Which data goes into the loss calculation? The sections below answer those questions in the order the script asks them.

The first line of the script selects a device. A device is the hardware that performs the computation. The source describes two options: a CPU, which is the general processor, and a GPU, which is the graphics processor supported by CUDA. The selection is written as follows.

The expression checks whether a CUDA-supported GPU is available by calling torch.cuda.is_available(). If that check returns true, the device is set to CUDA. If it does not, the device falls back to the CPU. The flowchart below shows the two branches and where the chosen device is applied next.

The source makes an important practical point: if the machine has a CUDA-supported GPU, the LLM trains on the GPU without any changes to the code. The same script therefore works on a laptop with no GPU and on a workstation with one. The selection happens at run time, so you do not need separate versions of the program.

Once the device is chosen, the script calls model.to(device). This moves the model's parameters onto the selected hardware. The source also states that the data must be loaded onto that same device. A model on the GPU with data still on the CPU would not be working on the same hardware, so the device value is passed explicitly into the loss function as well. The device setting is what keeps the model and the data together.

The next construct in the script is a with block. The source explains that the with torch.no_grad(): line wraps the two loss calculations and disables gradient tracking for efficiency, because the model is not being trained at this point in the script. The script is measuring loss, not learning from it.

Gradient tracking is the bookkeeping PyTorch normally does so that a model can be updated during training. It is necessary when weights are going to change. When the code is only measuring loss, that bookkeeping is not needed, so turning it off saves work. The source frames this as an efficiency choice; it does not change which data is measured or how the loss is defined.

Every line inside the with block runs with gradient tracking turned off. That means both loss values are computed under the same condition. If one loss were computed with tracking on and the other with tracking off, the two numbers would not be measured under identical conditions.

The two loss lines are nearly identical. Each one calls calc_loss_loader with a data loader, the model, and the device. Only the loader changes: train_loader is used for the training loss and val_loader for the validation loss. Because the function is the same in both calls, both numbers are computed the same way, which makes them directly comparable.

Read the diagram from left to right. Two loaders enter on the left, one carrying training data and one carrying validation data. The same model and the same device feed both calls. Each call produces one number, and those numbers are train_loss and val_loss. The source does not spell out the internal steps inside calc_loss_loader, so the diagram shows only its inputs and outputs. What matters for the lesson is that the same function, model, and device are used in both calls, and only the data changes.

The device line and the loader calls depend on each other. The source's logic implies several ways the script can fail or mislead when the pieces do not line up.

The two losses are computed by the same function, but they are measurements on different data. The table below sets out what stays the same and what changes between the training pass and the validation pass, using only what the source describes.

Because the conditions are held constant, any difference between the two numbers can be traced to the data rather than to a change in method. The source does not give numeric values for either loss, so no particular result should be assumed from the script alone.

The source's purpose for the two losses is to assess the quality of LLM-generated text during training. The training set loss shows how the model does on the text it learns from. The validation set loss shows how it does on text it is checked against. Neither number answers the purpose alone. The assessment comes from the pair.

The source states that the two losses are separate measurements that are read together, not interchangeable numbers. It also states that judging text quality from a single loss leaves out part of the evidence. Returning to the opening question: the training loss alone cannot show whether the model is learning something useful or only getting better at the text it has already seen, because it is measured on that same text. The validation loss supplies the second view.