Evaluating Large Language Model Responses

Fine-tuning adjusts a pre-trained LLM to a specific task, here converting active sentences to passive voice, using a single training call that sets the optimizer, epochs, and evaluation settings.

Programming

Deciding What a Healthy Run Looks Like

A pre-trained language model already knows a great deal about language, but it has not yet learned to follow the kind of instructions you care about. Fine-tuning closes that gap by adjusting the pre-trained model so that it fits a specific task or dataset. In this article the task is concrete: the model is trained on instruction data until it can convert an active sentence into its passive voice counterpart.

By the time training starts, the model and its data loaders are already prepared. What remains is a single training call that wires everything together. Because that call is where fine-tuning actually happens, it is the best place to begin. Treat it as a checklist of decisions: which optimizer updates the weights, how many passes over the data to make, how often to check progress, and what to ask the model to generate along the way.

What do you think happens?

You have read the configuration: two epochs, evaluation every five iterations, and two loss curves the run will produce. Before you see the output, what do you expect the training loss and the validation loss to do across the two epochs?

Reveal answer

Answer:

Both should decrease consistently across the two epochs.

The source reports consistently decreasing training and validation loss across the two epochs, and reads that steady decline as evidence that the model is learning. Committing to this prediction first makes the actual output meaningful when it arrives.

Reading the Training Call

Start by reading the optimizer line. It creates an AdamW optimizer over the model's parameters, which means the optimizer is the component that will update the model during training. Two of its settings are stated explicitly: the learning rate is 0.00005 and the weight decay is 0.1. Together with the optimizer choice, those two numbers make up the optimizer configuration for this run.

Which part of the training call sets the optimizer, learning rate, weight decay, epoch count, and evaluation schedule, and what does each one control?

The remaining arguments decide the shape of the run. The number of epochs is set to 2. The evaluation frequency is 5 and the evaluation iteration count is also 5, so the run checks its progress at regular intervals rather than only at the end. The call also receives a starting context, built by formatting the first validation instruction, which the model uses to generate a response while training is underway. The tokenizer is passed in so that text can be encoded and decoded during this process.

The call also returns three things you will read afterward: a list of training losses, a list of validation losses, and a count of tokens seen. The script records the start and end time around the call and reports the elapsed minutes. Timing does not change what the model learns, but it is useful when you compare runs.

Reading the Loss Curves

Once the run has finished, the losses are your main evidence. The source treats a steady decrease as the signal of improvement. What matters is the direction and the consistency of the change. A single lower number at one checkpoint is weaker evidence than a sequence of values that keeps falling as training continues.

How do training loss and validation loss change epoch by epoch, and at which point would they signal overfitting or a healthy run?

The loss curves that the run produces are where you test your prediction. Two kinds of loss are tracked, and you should read them side by side rather than in isolation.

Epoch Count and Overfitting

In this run the configuration uses exactly two epochs, and the source explains why that choice is reasonable. Because the losses keep decreasing and the responses are correct within those two passes, there is no need to add a third epoch. The source is explicit that continuing beyond that point is not essential and may even hurt, since it can lead to overfitting, where the model fits the training examples so closely that it generalizes less well.

Worked example

Judging a Hypothetical Run That Overfits

Suppose a longer fine-tuning run shows training loss falling epoch after epoch, while validation loss falls for one epoch and then rises. Is the run succeeding?

1
Check the training curve
Training loss keeps falling, which by itself looks like progress.
2
Check the validation curve
Validation loss stops improving and then rises, so the model is no longer doing better on held-out instruction examples.
3
Combine the two curves
The gap between the curves is widening. The model is fitting its training examples more closely while generalizing less well, which is the pattern the source associates with overfitting.
4
Decide on the epoch count
The epoch count should be chosen from the results. Here the evidence points to stopping earlier rather than adding passes.

Result

The run should not be judged a success on training loss alone. The rising validation loss signals overfitting, and the epoch count should be reduced.

Tracing a Run Through Config, Metrics, and Outputs

When a run looks wrong, do not jump straight to the loss numbers or to a single generated answer. Check three things in order: the configuration, the metrics, and the generated outputs. Each check narrows where the departure from expectation begins. If the configuration does not match the plan, the metrics and outputs are not measuring what you intended. If the configuration is right but the losses are not behaving as expected, the problem lies in the training behavior itself. If the losses look healthy but the responses are wrong for the task, the metrics are not telling the whole story.

In what order should I check the config, the loss values, and the generated responses, and which branch do I take when one of them looks wrong?

The generated responses are the most readable check in the whole run. The training call sends a starting context built from the first validation instruction, and the responses produced at the end of each epoch let you compare the model's answer directly against what the task asked for. For the active-to-passive task, a successful response is one that restates the sentence in passive voice, rather than repeating the active form or producing an unrelated sentence.

Mistakes When Judging a Run

Common mistake
Declaring success from the training loss alone
What it looks like
Training loss falls steadily, so the run is marked as finished.
Why it fails
Training loss measures fit to the data the model learns from. The source notes that a falling training loss with a non-falling validation loss is not the desired outcome.
Do this instead
Read the validation loss alongside the training loss, and then inspect generated responses.
Common mistake
Adding epochs by habit
What it looks like
Running a third or fourth epoch because more training seems better.
Why it fails
The source says epoch counts should be decided from the results. Continuing without evidence can lead to overfitting.
Do this instead
Keep the epoch count that the losses and responses support, and check whether more passes are actually needed.
Common mistake
Trusting one lower loss value
What it looks like
A single checkpoint shows a lower number, so the run is considered improving.
Why it fails
The source treats a sequence of values that keeps falling as stronger evidence than one lower number.
Do this instead
Look at the direction and consistency of the change across all checkpoints.
Common mistake
Skipping the configuration check
What it looks like
The losses look odd, and the optimizer settings are changed without first confirming what the call actually set.
Why it fails
The configuration is the first place to check whether the run matches the plan. Changing metrics or outputs without that check can hide the real cause.
Do this instead
Read the optimizer, learning rate, weight decay, epochs, and evaluation settings before interpreting results.

Practice: Diagnosing a Run

MEDIUM

A fine-tuning run is configured with the AdamW optimizer, learning rate 0.00005, weight decay 0.1, two epochs, and evaluation every five iterations. After the run, training loss and validation loss both decrease across the two epochs, but the response to the active sentence 'The dog chased the ball' is still in active voice. Using the diagnostic order, which check should you return to first, and what would you look for?

Hints

Start with the configuration, then the metrics, then the outputs.
Both losses decreasing is a healthy metric signal, so ask whether the outputs match the task.
The task is converting active sentences to passive voice; the response should show that change.

Your attempt (not saved)

Key Takeaways

Fine-tuning is driven by a single training call that sets the optimizer, the learning rate, the weight decay, the epoch count, and the evaluation schedule.
Training loss and validation loss should both decrease consistently for a healthy run; a falling training loss alone does not establish success.
Generated responses are the most direct check on whether the model now performs the target task, such as converting active sentences to passive voice.
Diagnose a run in order: configuration first, then metrics, then outputs, so that you can locate where the result departs from expectation.
Choose the epoch count from the evidence, since continuing beyond what the results support can lead to overfitting.

Related concepts

More concepts starting with E