Improving Model Performance and Tuning Caveats
Evaluation compares a model's predictions on a held-back test set against the known labels for those same flowers.
Why a Trained Model Needs Answers to Check Against
A trained k-Nearest Neighbors model is only useful if we can say how often it is right. Saying that it seems to work is not enough. Instead of guessing, we check its predictions against answers we already know. The evaluation step asks the model for predictions, then reports a single number that summarizes how well those predictions matched reality.
The goal of this article is to understand three things about that number: what it means, where it comes from, and what it does not promise. The number is only as trustworthy as the chain of steps that produced it, so we will follow that chain closely.
Where the Test Flowers Come From
The flowers used to evaluate the model are the test set that was set aside earlier. Those flowers were not used to build the model. That separation is what makes the evaluation meaningful. The model is being asked about flowers it has not already seen, and we know the correct species for each one because those labels were held back from the model during building.
Read the diagram from the top. The full dataset is divided into two portions. The training portion is used to build the model, and the test portion stays separate. Only the test portion is passed into the model to produce predictions. The known labels for those same test flowers are kept on the side and enter the process only at the comparison step. This is why the score can be trusted as a measure of performance on flowers the model has not already seen.
Lining Up Predictions with Labels
The evaluation has two inputs that must line up. The first is the model's prediction for each test flower. The second is the known label for that same flower. Each prediction is compared against its own label, and the results are then summarized. Position is what ties the two lists together: the prediction at position 3 must be compared with the label at position 3, because both describe the same flower.
Scoring five test flowers by hand
A model predicts the species of five test flowers. The known labels for those same five flowers are, in the same order, setosa, versicolor, virginica, virginica, setosa. The predictions are setosa, versicolor, virginica, versicolor, setosa. What is the accuracy?
- 1Pair each prediction with the label at the same position
Position 0 pairs setosa with setosa, position 1 pairs versicolor with versicolor, position 2 pairs virginica with virginica, position 3 pairs versicolor with virginica, and position 4 pairs setosa with setosa.
- 2Mark each pair as a match or a mismatch
Positions 0, 1, 2, and 4 are matches, so they are True. Position 3 is a mismatch, so it is False. The results are True, True, True, False, True.
- 3Count the correct predictions
There are 4 True values out of 5 flowers.
- 4Divide by the number of test flowers
The fraction is $\frac{4}{5}$, which equals 0.8. This means 80 percent of these five test flowers were predicted correctly.
The accuracy for these five test flowers is $\frac{4}{5} = 0.8$.
Accuracy as a Fraction of Correct Predictions
Accuracy is the fraction of flowers for which the right species was predicted. It is a proportion, so it lies between 0 and 1, and it is often read as a percentage when multiplied by 100.
The denominator is the number of test flowers, not the number of training flowers. The numerator counts only the test flowers whose predicted species matched the known label. A value of 0.97 therefore means the model was right for 97 percent of the test flowers. The source reads a test set accuracy of about 0.97 in exactly this way.
Reading the Evaluation Snippet Line by Line
The source shows two ways to obtain the test set accuracy. The first builds the comparison by hand: it checks equality between the predicted labels and the known labels, then takes the mean of the resulting True and False values. The second calls the score method of the trained model object, which performs the same test set accuracy computation internally. Both produce 0.97 in the source, so they agree on this model and this test set.
import numpy as np
y_pred = knn.predict(X_test)
matches = (y_pred == y_test)
accuracy = np.mean(matches)
print(accuracy)
print(knn.score(X_test, y_test))- 1Line 3 creates the predictions. The model is asked about the test flowers, producing one predicted species per flower.
- 2Line 4 is the comparison step. The == operator checks each prediction against the label at the same position and produces an array of True and False values.
- 3Line 5 produces the reported score.
np.meanaverages the True and False values, where True counts as 1 and False counts as 0, so the mean equals the fraction of correct predictions. - 4Line 8 is a second route to the same kind of number. knn.score performs the test set accuracy computation internally, so it does not expose the intermediate predictions and matches.
y_predholds the model's predictions, one per test flowery_testholds the known labels for those same flowers- matches is an array of True and False values from element-wise equality
np.meanturns the True and False values into a fraction
When you read any evaluation snippet, first find the line that holds the predictions, then the line that holds the labels, and then the operation that compares them and reduces the result to one number. The reported score is produced by the last of those steps, so that is where the number's meaning is decided.
Tracing a Surprising Score Back Through the Pipeline
It helps to see the evaluation as a flow. Test flowers go into the model, predictions come out, and those predictions meet the known labels in a comparison. The comparison results are then averaged into one fraction. Each stage hands a specific value to the next, so a surprising final number can be traced back through the stages to find the one that changed.
When a reported score is not what you expected, the flow gives you a checklist, and the checks run in order. First, confirm that the predictions came from the test set and not the training set. Second, confirm that the labels being compared are the labels that belong to those same flowers. Third, look at the prediction array itself. It should contain one entry per test flower, so a length mismatch points to a problem earlier in the pipeline. Only after those checks does it make sense to question the model.
Mistakes When Interpreting an Evaluation Score
Common mistake Scoring the model on the training flowers
What it looks likeThe model is evaluated on the same flowers it was built from, and the score looks excellent.
Why it failsThe model has already seen those flowers, so the score measures memory rather than performance on new flowers.
Do this insteadConfirm that the predictions were made on the held-back test set before trusting the number.
Common mistake Comparing predictions to the wrong labels
What it looks likeThe prediction list is compared with a label list whose order does not match the test flowers.
Why it failsEach prediction must be compared with the label for that same flower. A misordered label list produces a number that looks plausible but describes nothing real.
Do this insteadCheck that both lists describe the same flowers in the same order before comparing them.
Common mistake Assuming the number is wrong only because the code failed
What it looks likeNo error is raised, so the accuracy is assumed to be correct.
Why it failsA misaligned comparison runs without error and still returns a number between 0 and 1.
Common mistake Blaming the model before checking the pipeline
What it looks likeA low accuracy is reported as a model failure without looking at the prediction array.
Why it failsA length mismatch or a wrong label source is an earlier problem, and the model may be fine.
Do this insteadWork through the checklist: test inputs, then predictions, then labels, then the comparison.
What a 0.97 Score Does and Does Not Promise
The source interprets a test set accuracy of about 0.97 as meaning the model made the right prediction for 97 percent of the irises in the test set. That is a direct statement about the flowers that were scored. Under some mathematical assumptions, the same figure is taken as an expectation that the model will be correct about 97 percent of the time on new irises. For the hobby botanist in the source, that level of accuracy suggests the model may be trustworthy enough to use.
Practice: Finding Where a Wrong Score Came From
A colleague reports that a model's accuracy on five test flowers is 0.2, which seems far too low given how well the model performed during development. The predictions are setosa, setosa, setosa, setosa, setosa, and the known labels for the same five flowers are setosa, versicolor, virginica, setosa, versicolor. Work out the accuracy by pairing each prediction with its label at the same position. Then decide which stage of the pipeline is most likely to be at fault.
Hints
- Pair the two lists by position, not by value. Write out the pairs first.
- Count how many pairs match, then divide by the number of test flowers.
- The prediction array here is suspicious: a list of identical values across five different flowers is a clue about which stage to inspect.
Key Takeaways
- Evaluation compares a model's predictions on a held-back test set against the known labels for those same flowers, so the model is judged on flowers it has not already seen.
- Accuracy is the fraction of correct predictions, and it is computed either by comparing arrays and taking the mean or by calling the model's score method, which gives the same result.
- Predictions and labels must line up position by position; a misaligned comparison still runs without error and returns a meaningless number.
- When a score looks wrong, trace the flow from test inputs to predictions to the label comparison, checking the stages in that order before questioning the model.
- A test set accuracy such as 0.97 describes the flowers that were scored; extending it to new flowers depends on stated assumptions.