Evaluating Fine-Tuned Language Models
Instruction fine-tuning improves an LLM's ability to follow instructions and generate desired responses.
Why a Trained Model Still Needs Checking
Suppose you give a language model the request 'Convert 45 kilometers to meters.' You do not want a loosely related continuation of the text. You want the model to answer the request. Instruction fine-tuning is the process built for this purpose. Its aim is to strengthen an LLM's ability to follow an instruction and produce the response a person actually wants.
Fine-tuning changes a model's behavior, but a changed model is not automatically a better one. After training, you still have to find out how well the model performs on the task you care about. That question is the subject of this article: how a fine-tuned model is evaluated once training has finished.
What do you think happens?
Before evaluating anything, which of the three stages of the process must run first?
Reveal answer
Stage 1, dataset preparation
Each stage depends on the one before it. There is no dataset to train on until stage 1 is complete, and nothing to evaluate until stage 2 has produced a trained model.
The Three Stages and Nine Steps
The process is organized into three stages. Stage 1 prepares the dataset that will be used for training. Stage 2 loads a model and fine-tunes it on that dataset. Stage 3 evaluates the fine-tuned model. These stages are carried out as nine steps in order.
Stage 1 begins with the data. Its job is to obtain the instruction dataset, look at it, and arrange it so that a training process can consume it. The source describes this stage as downloading, formatting, batching and loading the instruction-response pairs. The dataset used in this process contains 1,100 such pairs.
Stage 2 applies the prepared dataset to a model. It has two actions. The fourth step loads a pretrained LLM, which establishes the starting model. The fifth step instruction fine-tunes that LLM, adjusting it using the instruction-response pairs. The sixth step inspects the loss, which lets you check how training is progressing as the adjustment runs. This is the point where the data from stage 1 actually changes the model's behavior.
Stage 3 asks whether the change was worthwhile. The seventh step extracts responses by running the fine-tuned model to produce its answers. The eighth step is qualitative evaluation, meaning reading and judging those answers. The ninth step scores the responses, turning the judgment into a score. Qualitative reading comes before scoring so that the numbers are attached to answers someone has already examined.
Following One Instruction Through the Pipeline
To see how the stages connect, follow one instruction through the process. Fine-tuning data is made of examples, and each example pairs an instruction with a response that fulfills it. The source offers a synonym request as one such pattern: for the instruction asking for a synonym, a good response names a word such as 'radiant.' The instruction used below, 'Provide a synonym for bright,' is an original illustration built from that same pattern. It is not a result the source reports.
Tracing 'Provide a synonym for bright'
Describe what each stage contributes to this single instruction.
- 1Stage 1 contributes a target
The instruction is stored as one pair with a response that fulfills it, such as a word like radiant. The pair is formatted and batched so training can consume it.
- 2Stage 2 contributes a change
The pretrained model is fine-tuned on pairs like this one, so its outputs move toward the targeted answers. Loss is inspected to check that training is progressing.
- 3Stage 3 contributes a judgment
The fine-tuned model is run on a test instruction of this kind, and its response is extracted. Someone reads the response, and then a score is assigned to it.
The instruction moves from a raw request, to a training pair, to a changed model, to a judged and scored response. Each stage adds something the next stage uses.
How Responses Become One Score
Evaluation is a process that produces a number from a collection of model outputs. Each step hands a slightly different collection to the next step, so the final number only makes sense if you know what each step did to the collection.
Evaluation starts with a test set: a set of inputs the model was not trained on. The fine-tuned model is run on each item in the test set, and its responses are collected. At this point the collection holds raw responses, and nothing has been judged yet.
The next step is scoring. Each response in the collection receives a score, so the collection changes from a list of text responses into a list of numbers attached to those responses. The source gives a concrete tool for this step: the Ollama application, paired with an 8-billion-parameter Llama model, can automatically score the fine-tuned model's responses on the test set. The source also notes that another LLM can serve as the scorer.
Notice that the number of items shrinks from many to one at the last step. The test set may contain many examples, but the reported performance is one number. Any question about a single example must be answered by going back to the scored collection, before the averaging took place.
Roles of the Test Set and the Scoring Step
The two parts of evaluation do different jobs. The test set supplies inputs to the model, and those inputs were kept apart from training. The scoring step does not supply inputs. It consumes the model's outputs and produces a judgment for each one.
| Aspect | Test set | Scoring step |
|---|---|---|
| What it contains | Inputs the model was not trained on | The model's responses to those inputs |
| Direction of data | Flows into the model | Flows out of the model and into a score |
| State it leaves behind | Unchanged inputs | Each response paired with a number |
| Example from the source | Items the fine-tuned model answers | An 8-billion-parameter Llama model run through Ollama, or another LLM |
How the test set and the scoring step differ within evaluation
Mistakes When Reading an Average
An average summarizes the test set, but it hides the spread of individual scores. Two models can differ greatly in response quality and still share the same average. The diagram below uses a hypothetical scale to show how this can happen.
Common mistake Treating the average as the full picture of quality
What it looks likeModel A and Model B both average 5 on the hypothetical scale, so they look identical.
Why it failsThe average discards the spread. Model B may contain responses that are very poor alongside responses that are very good.
Do this insteadKeep the per-item scores when you need detail, and go back to the scored collection before the averaging step.
Common mistake Reading a single number as a verdict on one example
What it looks likeConcluding that one specific instruction was answered well because the overall average is high.
Why it failsThe final state holds one value for the whole test set. It says nothing specific about any one response.
Do this insteadLook at the scored responses for the individual items you care about.
Common mistake Scoring before anyone has read the responses
Why it failsThe source places qualitative reading before scoring so that numbers are attached to answers someone has already examined.
Do this insteadRead and judge the extracted responses first, then score them.
Tracing the Evaluation Yourself
A fine-tuned model is run on a test set of five instructions and produces five responses. Write, in order, the state of the evaluation after the responses are extracted, after each response is scored, and after the scores are averaged. For each state, say what kind of object the collection holds and how many values it contains.
Hints
- Think about which step runs the model and which step attaches a number.
- The test set is the starting collection of inputs, not the outputs.
- The final state should contain a single value.
Key Takeaways
- Instruction fine-tuning aims to strengthen an LLM's ability to follow an instruction and produce the response a person actually wants.
- The process has three stages, carried out as nine steps in order: dataset preparation, model setup and fine-tuning, and evaluation. Each stage depends on the one before it.
- Evaluation runs the fine-tuned model on a test set of inputs it was not trained on, collects its responses, scores each one, and averages the scores.
- Each step changes the collection, from inputs to raw responses to scored responses to a single average, and the final number hides the spread of individual scores.