Preparing Instruction Datasets for Fine-Tuning
A baseline evaluation runs a pretrained LLM on an instruction-following task before any fine-tuning, so later improvements have a reference point.
Why Measure Before Training
Before we improve a language model, we need to know where it starts. This lesson looks at one question: how well does a pretrained LLM follow an instruction when it is used as it came, with no additional training for the task? The answer gives us a baseline. A baseline is a reference point that we measure once, so that later changes can be judged against it.
A baseline evaluation runs a pretrained LLM on an instruction-following task before any fine-tuning, so that later improvements have a reference point. An instruction-following task asks the model to produce a response that does what the instruction says, such as converting an active sentence to passive voice.
The check itself is simple. We take one example from the validation set, send the model its instruction, and compare what the model writes with the response we expect. The gap between those two is what we study. Keep that gap in mind, because the rest of the lesson is about reading it accurately.
The Instruction-Following Task
The validation example used here asks the model to convert a sentence from active voice to passive voice. The instruction is: Convert the active sentence to passive: 'The chef cooks the meal every day.' A correct response would rewrite the sentence so that the meal becomes the grammatical subject and the chef appears in a passive construction.
This task is well suited to a first assessment because a person can easily recognize whether a response is correct or incorrect. When the target behavior is that clear, any gap between the model's output and the expected response is easy to see and to describe.
Reading the Formatted Prompt
The assessment begins by fixing a random seed, selecting the first validation example, formatting it into a prompt, and printing that prompt so we can read exactly what the model will see. Printing matters because the model receives the formatted text, not the raw data record.
The printed prompt follows a template. It opens with a sentence that introduces the task, then places the instruction under an Instruction label. The instruction text is what the model is supposed to act on. The diagram below traces the prompt from its first token to its last, so you can see where each section begins.
Predicting the Model's Reply
Before we look at the model's output, decide what you expect. The prompt asks for a passive-voice rewrite of a simple active sentence. A model that has only been pretrained on general text may be continuing a document rather than carrying out the instruction. Choose the reply you think is most likely.
What do you think happens?
Which reply is most likely from a pretrained model given this prompt?
Reveal answer
The original sentence repeated under the Response label, followed by a new Instruction label
The model has learned the surface template of the prompt, so it writes a Response section. It does not perform the transformation, and it continues the pattern by starting another Instruction label.
What the Output Gets Wrong
The model's output appears after the Response label. Read it carefully, because the wording of the output matters more than its length. The output has two parts that deserve attention. First, it does contain a Response section, so the model has learned the surface format of the template. Second, the body of that section is the original sentence, The chef cooks the meal every day, and the output then begins a new Instruction label. Part of the instruction text, Convert the active sentence to passive, is also carried into the output as the model continues the pattern.
- Format: the output contains a Response section, so the template is partly reproduced.
- Relevance to the task: the body repeats the input sentence instead of rewriting it in passive voice.
- Completeness: the requested conversion is never performed, and part of the instruction text is echoed back.
- Stopping point: the output does not end after the answer; it begins a new Instruction label.
The comparison leads to one conclusion. The model is not yet capable of correctly following this instruction. It produces the form of a response without performing the transformation the instruction requires. That is the gap the evaluation was designed to reveal, and it is the reason the next step is fine-tuning, which aims to improve the model's ability to comprehend and respond to instructions like this one.
Mistakes in Reading a Baseline
Common mistake Treating the Response label as proof that the model answered
What it looks likeThe output contains a Response section, so the model seems to have followed the instruction.
Why it failsThe label only shows that the model learned the layout of the template. The body of the section still has to be checked against the instruction.
Do this insteadCompare the content under the Response label with what the instruction asks for, not just whether the label appears.
Common mistake Judging the output by its length
What it looks likeA long reply looks more capable than a short one.
Why it failsIn this example the output is not useful merely because it has text; the wording determines whether the task was done.
Do this insteadRead the wording and check whether the requested transformation is present.
Common mistake Reading the baseline as the final result
What it looks likeConcluding that the model can never do the task after seeing the baseline output.
Why it failsThe baseline is a single measurement taken before any training for the task. It is a starting reference, not a verdict.
Do this insteadRecord the baseline behavior so that a later run of the same kind of instruction can be measured against it.
Common mistake Forgetting to print the formatted prompt, which hides exactly what the model receives
The Workflow Around the Baseline
The baseline check sits between the pretrained model and fine-tuning. The steps below show the order in which the work happens, and where each measurement is taken.
When fine-tuning is applied later, the same kind of instruction can be run through the model again, and the result can be measured against the starting behavior recorded here. The value of the baseline comes from that comparison.
Checking Your Reading of a Prompt
Using the template described in this lesson, a formatted prompt contains a task introduction, an Instruction label, the instruction text Convert the active sentence to passive: 'The dog chases the ball.', and then a Response label with nothing after it. Which part of this prompt is the model meant to write, and where does that writing begin?
Hints
- Look for the label that marks the end of the instruction section.
- The model supplies the text that follows the Response label.
Key Takeaways
- A baseline evaluation measures a pretrained LLM on an instruction-following task before any fine-tuning, giving later improvements a fixed reference point.
- The formatted prompt opens with a task introduction, places the instruction under an Instruction label, and ends at a Response label where the model's answer is expected.
- A pretrained model may reproduce the Response format while failing to perform the requested transformation, repeating the input and continuing the template with a new Instruction label.
- Judge a baseline by comparing the model's output with the expected response, then rerun the same kind of instruction after fine-tuning to measure progress.