Model Evaluation and Text Generation

The evaluate_model function measures training and validation losses after model updates, so training progress can be judged rather than assumed.

Programming

Why a Falling Loss Is Not Enough

Training a language model means repeatedly updating its weights and watching the loss. A falling loss is encouraging, but it does not by itself answer the question that matters: does the training actually improve the model? Answering that question requires a separate measurement, taken after each model update, of the loss on both the training data and held-out validation data.

What do you think happens?

If the training loss keeps going down after every update, is that enough evidence that the model is improving?

Yes, a lower training loss always means the model has improved
No, a falling training loss alone does not answer whether training really improves the model

Reveal answer

Answer:

No, a falling training loss alone does not answer whether training really improves the model.

The course source states that a falling loss is encouraging but does not by itself settle the question. A separate measurement on both training data and held-out validation data is needed after each model update.

Where evaluate_model Fits in Training

In this course, the separate measurement step is handled by a function named evaluate_model. It takes four inputs: the model, a training data loader, a validation data loader, and the device the computation runs on. It also takes an eval_iter value that controls how many batches are used. It returns two numbers: a training loss and a validation loss.

The function sits inside the training cycle. The cycle repeats a model update and then calls evaluate_model, so each evaluation reports the state of the model after that update. This placement is what allows training progress to be judged from measured numbers instead of assumed from the fact that updates are happening.

Key point

evaluate_model does more than compute two numbers. It changes how the model behaves for the duration of the calculation, and the rest of this article explains those changes and why each line exists.

What Evaluation Mode Changes

The function places the model in evaluation mode before any loss is computed. Evaluation mode is a temporary state, not the model's permanent condition. Two separate behaviors are affected: dropout, which belongs to the model's layers, and gradient tracking, which belongs to the bookkeeping of the computation. Keeping these two ideas apart makes the function much easier to read.

The model starts in training mode, moves into evaluation mode when the function calls model.eval(), and returns to training mode when the function calls model.train() before it finishes. Dropout is active in training mode and switched off in evaluation mode.

Behavior	What it is	Why it is changed during evaluation
Dropout	A behavior of the model's layers	Switched off so the same input gives the same output across repeated evaluations
Gradient tracking	Bookkeeping of the computation	Switched off because evaluation only needs the loss value, not the gradients used to update weights

The two effects of evaluation mode, as described in the course source

Tracing evaluate_model in Order

The order of the steps is significant. The mode is set before any loss is computed, so both losses are measured under the same conditions. The mode is restored only after both losses have been computed, so the switch back to training happens at the end of the function, not in the middle of the measurement. The flowchart below follows the function in execution order.

The function sets evaluation mode first, computes the training loss and then the validation loss inside the no-gradient context, restores training mode, and returns both numbers. A faulty step shows up at the first point where the measured conditions stop matching what this order requires.

python

def evaluate_model(model, train_loader, val_loader, device, eval_iter):
    model.eval()
    with torch.no_grad():
        train_loss = calc_loss_loader(train_loader, model, device, eval_iter)
        val_loss = calc_loss_loader(val_loader, model, device, eval_iter)
    model.train()
    return train_loss, val_loss

def evaluate_model(model, train_loader, val_loader, device, eval_iter):
    model.eval()
    with torch.no_grad():
        train_loss = calc_loss_loader(train_loader, model, device, eval_iter)
        val_loss = calc_loss_loader(val_loader, model, device, eval_iter)
    model.train()
    return train_loss, val_loss

Notes on the code

model.eval() puts the model in evaluation mode, which disables dropout
torch.no_grad() turns off gradient tracking for the enclosed loss calculations
Both losses use the same model, device, and eval_iter, so they are measured under the same conditions
model.train() restores training mode after both losses are computed
The argument order shown for calc_loss_loader is illustrative; the source pack does not specify it

Worked example

Locating a faulty evaluation step

Two evaluations of the same model report different validation losses even though the data has not changed. Using the execution order of evaluate_model, where should the investigation start?

1
Check the mode at the start
The first step is model.eval(). If dropout were still active, repeated evaluations of the same input could differ, so this is the earliest point where unstable results would come from.
2
Check the gradient context
Next, confirm that both loss calculations sit inside the no-gradient context. This step affects overhead rather than the loss value itself, so it is less likely to explain changing numbers, but it is still part of the expected order.
3
Check that both losses share conditions
Confirm that the training and validation calls receive the same model, device, and eval_iter. If one call ran under different conditions, the two losses would not be comparable.
4
Check the restoring step
Finally, confirm that model.train() runs after both losses are computed. If it ran too early, the validation loss would be measured while the model was back in training mode.

Result

Working through the function in execution order finds the first step that does not behave as intended. The investigation starts at model.eval() because that step controls whether the model's behavior is stable from one evaluation to the next.

Training Loss Versus Validation Loss

The pairing at the core of evaluate_model is a single measurement taken on two different datasets. The training loss shows how the model performs on data it has learned from. The validation loss shows how it performs on data kept aside for checking. Because both are computed under the same conditions, the two numbers can be compared directly.

Both losses come from the same measurement procedure with the same model, device, and eval_iter. They differ only in which data is used.

Comparing the two losses after each model update is what allows the training process to be judged. The source supports this comparison as the purpose of the function; it does not specify a rule for interpreting every possible pattern in the numbers, so the reading of any particular gap should be made with care.

Mistakes That Hide Evaluation Problems

Common mistake
Treating the training loss alone as proof that the model is improving
What it looks like
Training loss falls after every update, so the training is assumed to be working.
Why it fails
The source states that a falling loss is encouraging but does not by itself answer whether training improves the model.
Do this instead
Compare the training loss with the validation loss, which is measured on held-out data after each update.
Common mistake
Assuming the model stays in evaluation mode after the function returns
What it looks like
Code that runs after evaluate_model expects dropout to be off.
Why it fails
The function calls model.train() before it returns, so the model goes back to training mode. Any code that runs afterward sees the mode the function left it in.
Do this instead
Remember that evaluation mode is temporary and that model.train() restores the training mode at the end.
Common mistake
Expecting both losses to be measured under different conditions
Why it fails
Both losses use the same model, device, and eval_iter, which is what makes them comparable.
Do this instead
When two losses are compared, confirm that the same conditions were used for each.
Common mistake
Restoring training mode before both losses are computed
Why it fails
The source says the mode is restored only after both losses have been computed, so the measurement stays under the same conditions.
Do this instead
Keep the restoring step at the end of the function, after both calls to calc_loss_loader.

Practice Tracing the Function

MEDIUM

A colleague's version of evaluate_model removes the model.train() call at the end and otherwise keeps the same steps. Trace the function in execution order and explain which later code would be affected, and what the model's mode would be when that code runs.

Hints

Look at what the source says the function leaves the model in, and what code that runs afterward would see.
Consider whether the mode change in this version is temporary or permanent.

Your attempt (not saved)

Key Takeaways

evaluate_model measures a training loss and a validation loss after model updates, so training progress can be judged rather than assumed.
Calling model.eval() disables dropout so that repeated evaluations give stable results, and the no-gradient context turns off gradient tracking because evaluation only needs the loss value.
Both losses are computed under the same model, device, and eval_iter, and the function ends by calling model.train() to restore training mode.
When an evaluation looks wrong, walk the function in execution order and find the first step that does not behave as intended.

Related concepts

More concepts starting with M