The Three-Stage Process for Instruction Fine-Tuning an LLM
Instruction masking replaces the instruction and input tokens with the -100 mask, and Exercise 7.2 asks you to evaluate whether that change has a positive effect on model performance.
Where This Experiment Fits
This article covers a small experiment that sits at the end of a chapter on instruction fine-tuning. By the time you reach it, you have already built an InstructionDataset class, written a custom_collate_fn function to batch instruction examples, and fine-tuned a model. The experiment asks for one more thing: change which tokens count during training, then judge whether that change helps. Think of it as a controlled test layered on top of a pipeline you already understand.
The chapter presents instruction fine-tuning as three stages: preparing the dataset, fine-tuning the LLM, and evaluating the LLM. Each stage contains numbered steps. The experiment touches two of these stages at once. Its masking change lives in the data that flows through the loaders, and the loaders are the last thing built in the first stage. So to understand the experiment you need to understand both the masking idea and the handoff from prepared data to batches.
The Three Stages and Where Data Loaders Sit
Read the workflow from left to right. Stage 1 prepares the dataset. Stage 2 fine-tunes the LLM. Stage 3 evaluates the LLM. Data loader creation is the closing move of stage 1. It is the point where prepared data stops being a collection of examples and becomes a stream of batches that the fine-tuning stage can consume. Everything in stage 2 depends on that handoff.
Masking Instruction and Input Tokens
Instruction masking replaces the instruction and input tokens with the -100 mask. The response tokens keep their real token IDs. The masking is applied to the training data, so the change affects which tokens count during training. The exercise asks you to evaluate whether this change has a positive effect on model performance.
The source names the -100 value as the mask. It does not spell out in this passage how the training code uses that value internally, so when you run the experiment, check the chapter's code to confirm the exact behavior rather than relying on an assumption about it.
Tracing One Example Through Masking
Before you run anything, trace a single training example by hand. The example below is illustrative and uses generic labels rather than real tokens. Each position in the example has an index. Your job is to decide which positions get the -100 mask and which keep their real token IDs.
Applying the mask to one example
A training example has three instruction positions, one input position, and three response positions. Which positions does masking change?
- 1Identify the masked categories
Masking replaces the instruction and input tokens. Those are the first four positions in this example.
- 2Replace those positions with -100
Positions 0 through 3 now hold the -100 mask instead of their real token IDs.
- 3Keep the response positions
Positions 4 through 6 are response tokens. They keep their real token IDs and are not masked.
Four positions are replaced with -100, and three response positions keep their real token IDs.
From Dataset to Batches
The two pieces play different roles. The InstructionDataset object holds the instruction examples. The custom_collate_fn function describes how individual examples are combined into a batch. The PyTorch data loader connects the two. It handles shuffling and batching, so the training code can simply request the next batch. Notice the division of labor: the dataset stores, the collate function shapes, and the loader delivers.
Data loaders are created for the training, validation, and test sets. Each split gets its own loader, so the same batching logic serves both fine-tuning and evaluation. Because the loaders are built once per split from the same collate logic, the batches that reach training and the batches that reach evaluation are shaped the same way.
Testing Whether Masking Helps
Because the exercise is an experiment, treat it as a sequence of decisions. Each step gives you a result you can compare against the step before it, so you can tell whether masking made a difference rather than guessing. The fair test changes masking as its only variable and compares the result against the earlier fine-tuned run.
- 1Keep the earlier fine-tuned run as your baseline and record how it performed.
- 2Change only the masking: apply the -100 mask to the instruction and input tokens in the training data.
- 3Keep everything else the same, including the dataset, the
custom_collate_fnbehavior, the data loaders, and the training settings. - 4Fine-tune the model on the masked data to produce a new run.
- 5Compare the masked run against the earlier baseline to judge whether the change has a positive effect on model performance.
| Aspect | Earlier fine-tuned run | Masked run |
|---|---|---|
| Instruction and input tokens | Keep their real token IDs | Replaced with the -100 mask |
| Response tokens | Keep their real token IDs | Keep their real token IDs |
| Other variables | Baseline settings | Same as baseline |
| Question answered | Baseline performance | Did masking have a positive effect? |
The only intended difference between the two runs is the masking of instruction and input tokens.
Common Mistakes with Masking and Loaders
Common mistake Masking the response tokens along with the instruction and input tokens
Why it failsThe source specifies that instruction and input tokens are replaced with the -100 mask. Response tokens keep their real token IDs.
Do this insteadMask only the instruction and input positions, then check the masked example position by position.
Common mistake Changing more than one thing between runs
Why it failsA fair test changes masking as its only variable. If the dataset, collate function, or settings also change, you cannot attribute a result to masking.
Do this insteadHold everything else constant and compare only against the earlier fine-tuned run.
Common mistake Treating the InstructionDataset as the batching step
Why it failsThe InstructionDataset holds the examples. The
custom_collate_fndescribes how examples are combined into a batch, and the PyTorch data loader handles shuffling and batching.Do this insteadKeep the three roles separate: dataset stores, collate function shapes batches, loader delivers them.
Common mistake Creating a loader only for the training set
Why it failsThe source states that data loaders are created for the training, validation, and test sets, each with its own loader.
Do this insteadBuild one loader per split so that evaluation uses the same batching logic as fine-tuning.
Common mistake Assuming the experiment result without comparing against the earlier run
Practice the Masking Experiment
For a training example with two instruction tokens, one input token, and three response tokens, write out the list of positions that receive the -100 mask and the positions that keep their real token IDs. Then state which two runs you would compare to judge the masking change, and what single variable differs between them.
Hints
- Instruction and input tokens are the ones that get masked.
- The baseline is the earlier fine-tuned run, and the masked run is the only variable change.
Key Takeaways
- Instruction masking replaces the instruction and input tokens with the -100 mask, while response tokens keep their real token IDs.
- A fair masking experiment changes masking as its only variable and compares the result against the earlier fine-tuned run.
- The InstructionDataset holds the examples,
custom_collate_fndescribes how they are combined into batches, and the PyTorch data loader shuffles and batches them for training. - Data loaders are created for the training, validation, and test sets as the last step of the dataset preparation stage, which is the handoff into fine-tuning.