Evaluating Classification Models

Once a classification model has been trained, the next question is whether it works. A classifier takes an input and assigns it to a category, such as spam or not spam, or a diagnosis label. Each assignment is either right or wrong, and the simplest way to summarize that behaviour is to count how often the model is right. Evaluating a classification model involves calculating the classification accuracy. That single number is the starting point for judging a model, and this article covers what it means, how to compute it, and what it cannot tell you.

To measure accuracy you need two lists of the same length. The first list holds the model's predictions. The second holds the true labels, which are the correct answers for the same inputs. Accuracy works by pairing the two lists position by position: the prediction at position 0 is compared with the true label at position 0, the prediction at position 1 with the true label at position 1, and so on. Each pair is either a match, which counts as correct, or a mismatch, which counts as an error.

In this small example, the model is checked on five emails. Emails 1, 2, 4 and 5 have a prediction that matches the true label, so they count as correct. Email 3 is marked as changed because the true label is spam but the model predicted not spam, so it is an error. The diagram shows only one pair per email; the accuracy calculation depends on how many of these pairs match.

The formula is simply the definition written as a ratio. Count the predictions the model got right, divide by the total number of predictions it made, and the result is a value between 0 and 1. Accuracy is therefore a fraction, and it can also be expressed as a percentage by multiplying by 100.

Accuracy is a single summary number. It tells you how often the model was right, but it does not reveal which predictions were wrong or what kinds of errors the model made. The calculation never asks which emails were misclassified. Eighty percent would be the same whether the two errors were harmless messages or dangerous ones. This is the key limitation: a good accuracy value answers how many predictions were correct, not whether the mistakes matter.

Consider a mailbox of ten emails, where nine are not spam and one is spam. A model that always predicts not spam is correct on all nine ordinary emails, so its accuracy is 9 out of 10, or 90 percent. Yet it never identifies the spam email. Compare that with a second model that makes two false alarms, flagging two ordinary emails as spam, but catches the one real spam email. Its accuracy is 8 out of 10, or 80 percent, which is lower, even though it does the job the filter was built for.