Looking at Your Data Before Modeling

The goal of this chapter is a model that can predict the species of an iris from a new set of flower measurements. Building such a model is only half the job. Before anyone applies it to real measurements, we need to know whether it actually works and whether its predictions deserve trust. This article covers the first step in answering that question: deciding which data we use to build the model and which data we use to check it.

The intuition behind the first option feels reasonable. The model has seen these examples, answers them correctly, and so seems to have learned something useful. The trap is that the same behavior can come from two very different models. One has learned a general pattern. The other has memorized its inputs. On the training data alone, the two look identical. The rest of this article shows how the data is divided so that the difference can be seen.

The phenomenon behind this misconception has a name: overfitting. A model overfits when it remembers the training data instead of generalizing well from it. When it does, it reproduces the correct answer for every training point. That looks like success, but the model has not captured a pattern that carries over to measurements it has never seen.

The solution is to hold some labeled data back. We show the model examples it has not seen during building, for which we still know the correct labels. In practice, the labeled data we have collected, here 150 flower measurements, is split into two parts. One part builds the model and is called the training set or training data. The rest assesses how well the model works and is called the test set, test data, or hold-out set.

Rather than cutting the table by hand, scikit-learn provides train_test_split, which shuffles the dataset and splits it for you. It extracts 75% of the rows as the training set, together with the corresponding labels. The remaining 25% of the rows, with the remaining labels, becomes the test set. You do not decide which individual flowers go where; the function handles that.

The exact amount for each part is somewhat arbitrary. A test set holding 25% of the data is nonetheless a good rule of thumb, and it is the proportion the function uses by default. For the 150 flowers, 75% is 112.5 rows and 25% is 37.5 rows, so the counts that actually appear depend on how the function rounds. The next section works through the real shapes.

A simpler approach would be to take the first 75% of the rows for training and the last 25% for testing. The function does not do this. Before making the split, train_test_split shuffles the dataset using a pseudorandom number generator. The reason becomes clear once you consider how the iris data is ordered.

The data points are sorted by their label. If the last 25% were used as the test set, every point in it would carry the label 2. A test set containing only one of the three classes would tell us very little about how well the model generalizes across the species. Shuffling makes sure the test data contains examples from all classes.

In scikit-learn, data is usually denoted with a capital X and labels with a lowercase y. The convention is inspired by the standard mathematical formulation $f(x)=y$, where x is the input to a function and y is its output. The capitalization carries meaning. X is a two-dimensional array, a matrix with one row per flower. y is a one-dimensional array, a vector with one label per row.

Because train_test_split returns X and y together in one call, row i of X_train and row i of y_train describe the same flower. The pairing is held by the position in the arrays. If the two were split separately and then shuffled independently, the measurements would no longer line up with their labels, and the model would learn from wrong pairs. Splitting both in one call is what keeps them aligned.

Shuffling is random, which raises a practical problem: running the same code twice could produce two different splits. To get the same output every time, you provide the pseudorandom number generator with a fixed seed through the random_state parameter. With the seed fixed, the outcome is deterministic and the line always produces the same split. The source book fixes random_state whenever it uses a randomized procedure, and the same habit is worth adopting whenever you want results you can compare.

The four outputs are NumPy arrays. X_train holds 75% of the rows of the dataset and X_test holds the remaining 25%. The shape attribute reports how many rows and columns each array has, so it is a quick way to confirm that the split did what you expected. Each iris row has four measurements, so the feature arrays should be four columns wide and the label arrays should be one-dimensional.