Creating PyTorch Dataset Objects

Before a model can learn from spam messages, the text has to be turned into a form the training code can use. In PyTorch, that preparation is packaged into a dataset object. A dataset object holds prepared data for one split of the work: training, validation, or testing. This article follows one setup that creates three such objects from three CSV files, and it treats that setup as a small mechanism to trace step by step rather than a finished recipe to copy.

Instantiating a PyTorch dataset means calling a custom class such as SpamDataset with a CSV file name and keyword arguments, producing a dataset object. Keep that definition in mind as you read the rest of the article, because every line you study is an instance of that single action repeated three times.

The setup has three main parts. The first part is the imports: PyTorch, the Dataset base class from torch.utils.data, the tiktoken library, and the SpamDataset class, which is imported from a module named chapter06. The second part creates a single tokenizer object. The third part creates three dataset objects, one each for the training, validation, and test CSV files.

Reading the setup in this order matters. The imports make names available, the tokenizer is created once and stored under a name, and the three dataset calls then use that name. When you trace the code, track what each variable holds after each line. A variable that is named but not yet assigned cannot be passed to a call, so the order of lines is part of the meaning.

Creating a dataset object works the same way you would call any Python class. You write the class name and pass arguments in parentheses. The source describes three arguments. The first is the name of a CSV file, such as "train.csv". The second is max_length, a keyword argument. The third is tokenizer, another keyword argument that receives the tokenizer object created earlier.

The file name is the first argument and is passed positionally, so it is identified by where it sits in the call. The listing uses keyword arguments for max_length and tokenizer, so those two values are identified by their names. Keyword arguments name the parameter they fill, which makes the call read as a set of labeled settings rather than a bare list of values. When you later trace a problem, those names show which setting went where.

The source states what the three arguments are for, but it does not explain how SpamDataset uses them internally. The class is imported from chapter06 and its definition is not part of the listing. What you can rely on is the mapping from each call argument to its parameter name, and the fact that the file name is the source of the rows the dataset will prepare.

The listing creates one tokenizer with tiktoken.get_encoding("gpt2") and passes that same object to every dataset call. The tokenizer is therefore a single shared object, not three separate tokenizers. Each dataset receives a reference to the same tokenizer through its tokenizer argument, which keeps the three splits consistent with one another in how they turn text into inputs.

The diagram shows the path the source implies at a high level. A row of text is handled by the tokenizer, and the max_length setting governs the length the prepared item must reach through padding or truncation. The source does not detail the exact padding or truncation rules, so treat the middle step as the role of max_length, not as a full specification of the process.

The training dataset is created with max_length=None. The validation and test datasets are created with max_length=train_dataset.max_length. The listing reads a value from the training dataset object and reuses it for the other two splits. Reading an attribute from an existing object is a normal Python pattern: the dot followed by a name looks up a stored value on that object.

The source states that data loaders are created after the dataset objects have been instantiated. In this section's sequence, the three dataset objects come first, and the data loaders are built from them afterward. The dataset objects hold the prepared data, and the data loaders are the next step that depends on those objects existing.

The source text at this point does not include the code that builds the data loaders, so this article does not show that code. What you can rely on is the ordering: a dataset object must exist before a data loader can be created from it. If a loader were built before its dataset existed, the name it refers to would not yet be defined.