KNeighborsClassifier Class

Suppose you have a collection of labeled examples and a new data point whose label is unknown. One simple way to guess that label is to look at the examples that are most similar to the new point and let them decide. The k-nearest neighbors classification algorithm works this way. It makes a prediction for a new data point by considering its closest neighbor or neighbors in the training set.

The word neighbor refers to a training example that is close to the new point. The number of neighbors considered is the k in the algorithm's name. This article follows one complete path: how the algorithm uses neighbors to predict, how the KNeighborsClassifier class carries out that work, and how the accuracy of the resulting model is measured on the test set.

Follow a single prediction from start to finish. The new data point arrives first. The algorithm then measures how far that point is from each training example. From those distances it selects the k nearest training examples, which are the neighbors. Finally, it takes a majority vote among the labels of those neighbors, and that majority label is the predicted class.

Notice what the algorithm does not do. It does not first search for a rule that separates the classes. The training examples are consulted at the moment a new point arrives, and only the examples nearest to that point influence the decision.

In Python, the k-nearest neighbors algorithm is implemented in the KNeighborsClassifier class. This class contains two things: the algorithm that builds the model, and the algorithm that makes a prediction using that model. You do not write the neighbor search yourself. Instead, you create an object from the class and call its methods.

Think of the class as a toolkit with two jobs. One job turns training data into a model. The other job uses that model to label new points. Both jobs live inside the same object, so the model-building step and the prediction step are connected through that one object.

The fit method builds the model from training data. It takes two inputs: X_train, which holds the training examples, and y_train, which holds their training outputs, meaning the class labels that go with each example.

Because k-nearest neighbors does not need a separating rule in advance, the training step is about preparing the examples so that neighbors can be found later when predictions are requested. The examples and their labels are what the prediction step will rely on.

The source describes a workflow in which training data goes into fit to produce a model, and then the test set goes into score, which reports accuracy. The diagram below shows how the data moves through that path. Watch which set feeds which step, and notice where the accuracy number comes from.

The sequence below shows the order in which your program calls the methods. First you call fit with the training data. Then you ask the object to predict, and finally you call score with the test set data and test set labels.

The score method computes the model's accuracy when applied to the test set data and the test set labels. It does this by using the fitted model to make predictions for the test examples and comparing them with the true labels. The accuracy is the share of test examples the model labeled correctly.

An accuracy of 97% therefore means that, on the test set, the model's predictions matched the true labels for about 97 out of every 100 test examples. The statement is about the test set. It does not claim that the model is 97% correct on every possible new point, and it does not describe how the model performed on the training examples it was built from.