Preparing Datasets for Fine-Tuning

By the time fine-tuning begins, two large pieces of work are already finished. The first is a model that has been pre-trained and loaded into memory. The second is a dataset of instructions that has been cleaned, formatted, and turned into batches. Instruction fine-tuning is the step that connects these two pieces: the model is trained further, this time on instruction examples, so that it learns to respond to the kind of input the dataset describes.

This article looks at the moment just before that training step. The goal is to understand what has been prepared, what the fine-tuning step is allowed to rely on, and how to tell which part of the overall process you are looking at when something goes wrong. The work starts with a single setup listing that creates three dataset objects, so that is where we begin.

The listing that prepares the data has three main parts. The first part is the imports: PyTorch, the Dataset base class from torch.utils.data, the tiktoken library, and the SpamDataset class, which the listing imports from a module named chapter06. The second part creates a single tokenizer object with tiktoken.get_encoding("gpt2"). The third part creates three dataset objects, one each for the training, validation, and test CSV files.

A useful habit when reading this listing is to follow it line by line before reading any explanation. For each line, note what the variable is and what the call receives. The tokenizer is created once and that same object is passed to every dataset call, so all three splits are prepared with the same tokenizer.

Creating a dataset object in this section means calling the custom SpamDataset class the same way you would call any Python class: you write the class name and pass arguments in parentheses. The source describes three arguments. The first is the name of a CSV file, such as train.csv. The second is max_length, a keyword argument. The third is tokenizer, another keyword argument that receives the tokenizer object created earlier.

The listing uses keyword arguments for max_length and tokenizer. A keyword argument names the parameter it fills, so the call reads as a set of labeled settings rather than a bare list of values. The file name is passed positionally, as the first argument. Keeping the names visible makes it easier, when you later trace a problem, to see which setting went where.

The training dataset is created with max_length=None. The validation and test datasets are created with max_length=train_dataset.max_length. The listing therefore reads a value from the training dataset object and reuses it for the other two splits. Reading an attribute from an existing object is a normal Python pattern: the dot followed by a name looks up a stored value on that object.

The purpose of this pattern is consistency. If the validation and test datasets were created with a different maximum length than the training dataset, the splits could be prepared differently even though they come from the same data source. The source does not explain the internal logic inside SpamDataset that determines the max_length value, because the class is imported from chapter06 and its definition is not in the listing. What the listing does show is which value gets passed where, and that is what you can check when tracing a run.

The source states that data loaders are created after the dataset objects have been instantiated. In this section's sequence, the three dataset objects come first, and the data loaders are built from them afterward. The dataset objects hold the prepared data; the data loaders are the next step that depends on those objects existing.

The source text at this point does not include the code that builds the data loaders, so this article does not show that code. What you can rely on is the ordering: a dataset object must exist before a data loader can be created from it. If a loader were built before its dataset existed, the name it refers to would not yet be defined.

Instruction fine-tuning is organized as a three-stage process. Stage 1 prepares the dataset. Stage 2 fine-tunes the LLM. Stage 3 evaluates the LLM. The diagram below places the steps in order and groups each step under the stage it belongs to.

Notice where the fine-tuning work sits. Stage 1 and Stage 3 surround it. Stage 1 produces the batched data loaders that the training step will consume, and Stage 3 looks at the responses the tuned model produces afterward. Stage 2 is the only stage in which the model's weights are trained, and it contains three steps: loading the pretrained LLM, running instruction fine-tuning, and inspecting the modeling loss.

Fine-tuning does not begin from random weights. It continues training the model that already exists, which is the model loaded in the previous section. The data side is the instruction dataset prepared earlier in the chapter. That dataset was processed at the start of the chapter, so the fine-tuning step does not repeat that preparation work.

Fine-tuning also needs a way to compute loss and run training. The chapter already contains code for both, so the fine-tuning code does not rewrite them. It imports the two functions calc_loss_loader and train_model_simple from chapter 5. Four inputs meet at the single fine-tuning step: the pre-trained LLM, the instruction dataset, and the two chapter 5 functions that supply the loss calculation and the training procedure.

Suppose a run of the process produces a result you did not expect. The three-stage structure gives you a way to narrow the search. Ask first which stage the problem most likely belongs to. If the data loaders are wrong, the cause lies in Stage 1. If the model is loaded incorrectly or the loss and training steps misbehave, the cause lies in Stage 2. If the responses look wrong after training finishes, the cause may be in Stage 3, where responses are extracted and scored.