Baseline Models and Dummy Classifiers

Accuracy asks one question: across all samples, how often was the prediction right? It is a useful starting point, but it can hide what matters most in a binary classification problem. A model can look reasonable on accuracy while still failing on the rare class you actually care about.

Throughout this article, the working example is a 'nine vs. rest' dataset. The goal is to recognize the digit nine among all other digits. The nine samples are the minority class. That imbalance is what makes accuracy a poor guide here, and it is what makes precision, recall, and F1 useful.

This kind of model is called a dummy classifier. It is a trivial baseline: it ignores the input features and follows a fixed rule, such as always predicting the most frequent class. Its purpose is not to be good. Its purpose is to set a floor that any real model must clear, and to expose what a headline number like accuracy can conceal.

The confusion matrix breaks predictions into four counts: true positives, false positives, true negatives, and false negatives. Precision and recall are two of the most common ways to summarize those four counts, and both are built from the same pieces. A true positive is a positive sample the model correctly called positive. A false positive is a negative sample the model wrongly called positive. A false negative is a positive sample the model missed by calling it negative. A true negative is a negative sample correctly called negative.

The clearest way to remember the two metrics is to place each error on the predicted-versus-actual grid. False positives sit in the column of samples predicted positive, and precision is the metric that is limited by them. False negatives sit among the actual positives the model missed, and recall is the metric that is limited by them. True positives are the shared piece that both metrics count in their numerators.

The clearest evidence for the trade-off comes from two extreme models. Trace each one's behavior, then check the comparison below. The first model predicts every sample as positive. It cannot miss a positive, so it has no false negatives and its recall is perfect. But every negative becomes a false positive, and it has no true negatives, so its precision is very low.

The second model is the opposite. It predicts only the single data point it is most confident about as positive and labels everything else negative. If that one point really is positive, precision is perfect. But nearly all the positive samples are missed, so recall is very bad.

Because each metric shows only part of the picture, the source suggests summarizing them together. The F-score, also called the F-measure, is the harmonic mean of precision and recall. The variant most often used is the F1-score, which weights precision and recall equally.

The harmonic mean matters. Because it is not a plain average, the F1-score is pulled toward the smaller of the two values. A model cannot earn a high F1-score by excelling at only one of precision or recall. This is exactly the failure the extreme models exposed.

The source computes the F1-score for four models on the nine vs. rest test set. Before looking at any output, form your own expectation of how the four scores will compare. Then check the reasoning: the dummy classifier and the decision tree are separated clearly by F1, a gap that was not obvious when looking at accuracy alone. F1 captures what makes a good model in this case far better than accuracy does.

The most frequent-class model in the source gets an F1-score of 0.00, and the source reports an error message for it. The reason is that this model never predicts the positive class. Precision's denominator counts the samples predicted positive, and with no positive predictions that denominator is zero. The F-score's denominator therefore becomes zero too. Trying to divide by zero is the failure point, which is why the output is an error or a degenerate value rather than a meaningful score.

If you want a more complete summary, the classification_report function computes precision, recall, and F1-score together and prints them in a readable layout. The source uses this call for the most frequent model, with class names supplied for readability. The sample call below is a generated illustration of that pattern; variable names will match your own workflow.

Each class row reports precision, recall, and F-score with that class treated as the positive class. Support, the last numeric column, is the count of samples in that class according to the ground truth. The final row gives a weighted average of the per-class numbers, where each class is weighted by its sample count.