Other Evaluation Metrics for Classification

A classification model looks at an input and assigns it to a category. A spam filter might label an email as spam or not spam, and an image model might label a photo as a cat or a dog. Once a model has made a batch of these assignments, the obvious next question is how good those assignments were.

Evaluating a classification model means answering that question with a number. The number this article focuses on is classification accuracy. Before we compute anything, try to guess: if a model makes ten predictions and gets seven of them right, what share of its predictions were correct?

Accuracy is about correctness. Each prediction either matches the true label or it does not. To see this, line up the predictions and the true labels by position. The first prediction is compared with the first true label, the second with the second, and so on. Every position gives one of two outcomes: a match or a mismatch.

Consider five emails. The model predicted spam, not spam, spam, spam, and not spam. The true labels were spam, not spam, not spam, spam, and not spam. The diagram below pairs each position's prediction with its true label. The highlighted position is the one where they disagree.

Four of the five positions match and one does not. Counting those matches is the first step toward measuring accuracy.

The source describes the calculation in three steps. Count the correct predictions. Count the total predictions. Divide the first count by the second. The result is the fraction of predictions the model got right, and it can be expressed as a fraction or a percentage.

The diagram below follows the same five-email example through the calculation. It counts the correct predictions, counts the total, divides, and then converts the result to a percentage.

The fraction and the percentage describe the same quantity, but they are written differently. The fraction $\frac{4}{5}$ shows both the number correct and the total in one expression. The percentage 80% is a scaled version of the same share, which is convenient for comparison but hides the total unless you state it.

The source advises that accuracy should always be reported with its total in view, and that the fraction and percentage forms should be kept clearly distinguished. A higher accuracy means a larger share of the model's predictions were correct, and a lower accuracy means a smaller share were.

Accuracy treats every prediction equally: each one either counts as a match or as a miss. That makes it simple to compute, but it also means a single number can hide what happens to particular cases. Suppose a set of ten emails contains nine that are not spam and one that is spam. A model that always predicts not spam gets nine of ten right, so its accuracy is 90%. Yet it never catches the spam email at all.

Model A has the higher accuracy, but Model B is the one that actually finds the spam. This does not change how accuracy is calculated. It is a reminder that accuracy summarizes overall correctness and says little about which kinds of predictions the model gets right or wrong. When reading an accuracy score, ask what the model is being evaluated on, and whether the single number tells the whole story.