Train Test Split

Suppose you have a dataset and want to build a machine learning model from it. A natural question follows: how will you know whether the model is any good? If you judge the model only on the same data it was built from, you are measuring how well it fits examples it has already seen. That is not the same as measuring how well it will handle new cases. The train-test split is the standard way to keep those two questions apart.

This article follows one small Python program that classifies records from the iris dataset with the k-Nearest Neighbors algorithm. The program is short, but it contains every step a typical supervised learning workflow needs: split the data, fit a model, and evaluate it. As you read, keep one question in mind at each line: what new piece of information exists now that did not exist before? Tracking that question is how you find where a result went wrong.

The split happens before the model is trained. The order of events matters: the training set is the only part the classifier sees when it is fitted, and the test set is held back until evaluation. The split is therefore a deliberate decision about which data the model is allowed to learn from and which data is reserved for judging it.

The source passes random_state=0 to the split. The random_state parameter makes the random splitting reproducible: running the same code again produces the same split and therefore the same score. Without a fixed value, a different random arrangement of the data can lead to a different score, so the number you report depends on the split you happened to draw.

The k-Nearest Neighbors classifier is a classification algorithm. To predict the label of a new instance, it looks at the k training instances closest to that new instance and takes a majority vote among their labels. The label that appears most often among those neighbors becomes the prediction. The letter k is the number of neighbors considered.

The number of neighbors is passed as n_neighbors when the classifier is created, as in KNeighborsClassifier(n_neighbors=n). It is a hyperparameter: a setting you choose before training rather than a value the model learns from the data. The source example uses n_neighbors=1, which means each prediction simply copies the label of the single closest training point. With a larger k, the vote involves several neighbors, so the same training data can produce different predictions for different values of k. This matters at evaluation time, because the score you get depends on the settings you chose.

The program below reconstructs the steps the source describes. Read it as a sequence of state changes. Before each line, ask what information exists now that did not exist before.

The flow from the full dataset through the split, the fit step, and the score step is shown in the pipeline diagram below.

The score() method evaluates the model's performance on the held-out testing data. The result is an accuracy-style score, and the source formats it with print to two decimal places. A score of 0.97 means the classifier labeled about 97 percent of the held-out test instances correctly, under the conditions of this particular split and this value of n_neighbors.

The two parts have different jobs. The training set is the material the model is built from. The test set is held back and used afterward to check the model's performance on data it was not built from. The test set is described as containing data that is new to the model, which is exactly what makes it useful for judging generalization.

Notice that the test set reaches the evaluation step but does not feed into building the model. That separation is the whole point of the technique. If the test set were also used to build the model, it would no longer be data the model had not seen, and the evaluation would lose its meaning.