Fine-Tuning LLMs on Instruction Data
A baseline evaluation runs a pretrained LLM on an instruction-following task before any fine-tuning, so later improvements have a reference point.
Why Test a Model Before Training It
Fine-tuning is the process of training a model further so that it gets better at a task. Before any of that training happens, there is a question worth answering: how well does a pretrained LLM follow an instruction when it is used exactly as it came, with no additional training for the task? The answer is a baseline. A baseline is a reference point that is measured once, so that later changes to the model can be judged against it.
The check itself is simple. Take one example from the validation set, send the model its instruction, and compare what the model writes with the response we expect. The gap between those two is what we study. Keep that gap in mind as you read, because the rest of this lesson is about reading it accurately.
The Instruction-Following Task
An instruction-following task asks the model to produce a response that does what the instruction says. The model is judged on whether it performed the requested action, not on whether it produced some fluent text.
The validation example used here asks the model to convert a sentence from active voice to passive voice. The instruction reads: Convert the active sentence to passive: 'The chef cooks the meal every day.' A correct response would rewrite the sentence so that the meal becomes the grammatical subject and the chef appears in a passive construction. This task is easy for a person to recognize as correct or incorrect, which is exactly why it is useful for a first assessment. When the answer is clearly right or wrong, the gap between the model and the target is easy to see.
Reading the Formatted Prompt
Before the model sees the instruction, the instruction is placed inside a template. The assessment fixes a random seed, selects the first validation example, formats it into a prompt, and prints that prompt so you can read exactly what the model will see. Printing the prompt matters, because the model only ever receives the text itself, not the intention behind it.
The printed prompt follows a template with three parts you should be able to locate. It opens with a sentence that introduces the task. Next comes the instruction, placed under an Instruction label; this is the text the model is supposed to act on. Last comes a Response label. The prompt stops right after the instruction, with no answer written in. The model is expected to supply the text that follows the Response label, and that open space is where its behavior becomes visible.
Tracing the Baseline Pipeline
The baseline evaluation is a short sequence of steps. The instruction goes in, the model's output is collected, that output is compared with the expected response, and the result is recorded. The recorded result is what later fine-tuning will be measured against. Trace the steps in order, because each one depends on the one before it.
Predicting the Pretrained Model's Reply
What do you think happens?
The prompt asks for a passive-voice rewrite of 'The chef cooks the meal every day.' Before reading the output, what do you think a model that was only pretrained on general text will write after the Response label?
Reveal answer
The model repeats the original sentence, The chef cooks the meal every day, and then begins a new Instruction label, carrying part of the instruction text into its output.
A model that has only been pretrained on general text tends to continue a document rather than carry out the instruction. It has learned the surface format of the template, including the Response section, but it did not perform the transformation the instruction requires.
The model's output appears after the Response label. In the source example, the wording of that output matters more than its length. Two features deserve attention. First, the output does contain a Response section, so the model has picked up the surface format of the template. Second, the body of that section is the original sentence, The chef cooks the meal every day, and the output then begins a new Instruction label. Part of the instruction text, Convert the active sentence to passive, is also carried into the output as the model continues the pattern.
Comparing the Output with the Expected Response
Placing the model's output beside the expected response makes the gap concrete. The expected response restructures the sentence: the meal becomes the grammatical subject, and the chef appears in a passive construction. The model's output does none of this. It copies the input sentence into the Response section and then starts writing more template text.
| Aspect | Expected response | Pretrained model output |
|---|---|---|
| Instruction adherence | Performs the active-to-passive conversion | Does not perform the conversion |
| Content | Rewrites the sentence with the meal as subject | Repeats the original active sentence |
| Format | Answer given under the Response label | Response section present, but it continues into a new Instruction label |
| Completeness | Ends with the requested rewrite | Keeps generating template text after the answer |
Where the pretrained output falls short of the expected response
Why the Baseline Comes Before Fine-Tuning
The comparison leads to one conclusion. The pretrained model is not yet capable of correctly following this instruction. It produces the form of a response without performing the transformation the instruction requires. That is the gap the evaluation was designed to reveal, and it is the reason the next step is fine-tuning, which aims to improve the model's ability to comprehend and respond to instructions like this one.
The baseline's value comes from the comparison it enables. When fine-tuning is applied later, the same kind of instruction can be run through the model again, and the new result can be measured against the starting behavior recorded here. Without the baseline, an improved answer would have nothing to be compared with, and you could not show that the training changed anything.
Mistakes When Reading a Baseline
Common mistake Treating any fluent output as a correct answer
What it looks likeThe model writes a Response section, so it looks like it succeeded.
Why it failsThe baseline asks whether the model performed the requested action. A correct-looking format can hide a failure to transform the sentence.
Do this insteadCheck the body of the Response section against what the instruction asked for.
Common mistake Reading the prompt as the model's answer
What it looks likeMixing up the Instruction text with the text written after the Response label.
Why it failsThe prompt stops after the instruction. Only text produced after the Response label is the model's output.
Do this insteadDraw the boundary at the Response label and measure only what follows it.
Common mistake Assuming the baseline should be re-measured after every change
What it looks likeReplacing the recorded starting output with a newer run.
Why it failsThe baseline is a reference point measured once so later changes can be judged against it.
Do this insteadKeep the original recorded output and compare each later result to it.
Practice Reading the Gap
Suppose a pretrained model is given the same prompt and writes: The chef cooks the meal every day. Then it begins a new Instruction label. Identify three things in this output that show it did not follow the instruction. Then state where the Response label sits relative to the text you counted.
Hints
- Compare the body of the Response section with the rewrite the instruction asks for.
- Look at what the output does after the repeated sentence.
- The Response label separates the input from the output being measured.
Key Takeaways
- A baseline evaluation runs a pretrained LLM on an instruction-following task before any fine-tuning, giving later improvements a fixed reference point.
- The formatted prompt has a task introduction, an Instruction label followed by the instruction text, and a Response label with open space where the model writes its output.
- In the source example, the pretrained model produced a Response section but repeated the input sentence and carried part of the instruction into a new Instruction label, without performing the active-to-passive conversion.
- The gap between the model's output and the expected response is what the evaluation studies, and it motivates fine-tuning to improve instruction following.