Instruction Fine-Tuning Test Datasets

Once a model has been instruction-tuned, the next question is how well it follows instructions on examples it has not been trained on. A common approach is to take a held-out test set. Each entry in that set already contains an instruction, an optional input, and a reference output. The model is asked to answer every instruction, and the answers it produces are stored alongside the original entries so they can be compared later.

This article follows the practical loop that does that work. It generates a response for each entry, writes that response into the dataset, saves the result to a file, and confirms that the response landed where it should. The goal of a single pass is simple to state. The loop starts with an entry that holds an instruction, an input, and a reference output, and it must finish with a model answer attached to that same entry.

The loop reuses the same generate method that was used earlier in the chapter. The difference is scale: instead of generating for a single example, it runs once for each entry in the test set. The loop is driven by enumerate over test_data, so every iteration has both an index i and the entry itself. A tqdm progress bar wraps the loop, which is why the run reports how many of the 110 entries have been processed.

For one entry, the body of the loop always runs the same sequence of steps. It builds a prompt from the entry, converts that prompt into token IDs, generates a continuation, decodes the full output back into text, and cleans the text down to the answer. The cleaned answer is then written back into the entry. After that, a decision checks whether more entries remain. If they do, the body repeats for the next index. If not, the loop ends. The diagram below shows the order of operations for one entry and the decision that repeats the body.

The first two steps prepare the model's input. The code builds a prompt from the current entry using format_input. That prompt is then converted into token IDs with text_to_token_ids and moved to the target device. Those IDs are passed to generate, which is configured with three settings. It is told to produce at most 256 new tokens, to use the context length stored in BASE_CONFIG, and to stop when it reaches the end-of-sequence token whose ID is 50256.

The output of generate is a sequence of token IDs that contains the prompt followed by the newly generated continuation. The code converts the whole sequence back into text with token_ids_to_text. Because the prompt was part of what was fed in, the decoded string still begins with the prompt. The answer is therefore buried at the end of the string, and cleaning is the step that isolates it.

Cleaning has three operations. First, the code slices off the first part of the string whose length equals the length of the prompt text. Second, it removes the marker '### Response:' wherever it appears. Third, it calls strip to trim surrounding whitespace. The result is the response text alone. The diagram below shows which part of the decoded string is discarded and which part is kept.

The response is not returned from the loop or collected in a separate list. Instead, the code writes it directly into the entry it came from. It uses the index i to pick the correct dictionary inside test_data and stores the cleaned text under the key model_response. Because the dictionary is modified in place, the change is visible everywhere test_data is used afterward, including the file-writing step.

After the loop finishes, the updated test_data is written to a file named instruction-data-with-response.json. The file is opened in write mode 'w', and json.dump serializes the list into that file using an indent of 4 spaces. The indentation is there so the saved structure is easy for a person to read. The file holds a list, and each element of that list is one entry dictionary.

Each entry in the file keeps its instruction, input, and reference output, and it also carries the new model_response field. This makes the file a complete record of the test run rather than a bare list of answers. The diagram below shows how the fields nest relative to one another.

The file is not written inside the loop. The write step comes after the last entry has been processed, so the complete dataset, with every model_response filled in, is serialized in one operation. Writing once at the end means the file reflects the finished run. A partially filled file is never produced by the loop body itself.

Once the run has finished, it is worth inspecting one entry directly rather than trusting that the loop worked. The source examines a single entry from the updated test set. That entry contains the original instruction, the input, the reference output, and a model_response field whose value is the model's answer. Seeing all four keys together confirms two things: the response was added to the correct dictionary, and the original fields were not disturbed.