Multiclass Classification Basics

A multiclass classifier does more than name one answer for each sample. For every sample it can report how strongly it leans toward each possible class. When you have three or more possible labels, such as three flower species, the model's output is therefore a whole table of scores rather than a single label. Learning to read that table is the first skill this lesson builds.

This lesson starts from running the code and asking what the returned structure tells you, then builds the rules that explain it. Before you read anything else about the output, check two things: how many rows and columns the printed array has, and what the numbers add up to across each row. The first check tells you the layout. The second tells you what kind of quantity the numbers are.

predict_proba returns an array with one row per sample and one column per class, so its shape is (n_samples, n_classes). Each row belongs to one sample and each column belongs to one class. The entry at a given row and column is the classifier's probability that this sample belongs to that class. Because the columns are classes, a single row contains a full set of class probabilities for one sample.

The rows always sum to 1, which means each sample's probabilities are spread across its classes and together cover all the possibilities the model considers. The layout is always (n_samples, n_classes), and that holds for every classifier that uses this method, including the binary case. Keep this in mind, because the binary case of decision_function is the one place where the shape looks different.

In this equation $p_{ik}$ is the entry in row i and column k, and K is the number of classes. Read the equation as a rule about one row: fix the sample i, then add up its entries across all K columns, and the total is 1.

Follow the picture for one sample. Every column is a class, and the three values together describe how the model spreads its belief for that single sample. The highlighted column holds the largest value, which is the one the model leans toward most. Finding that column is the next step.

Once you know each row is a set of class probabilities, the predicted class is the column holding the largest value in that row. NumPy's argmax with axis=1 finds that column position for every row at once. The result is a position, not a class name, so it still needs a mapping step before you have a label.

The mapping step uses the classes_ attribute. The classes_ attribute stores the class labels in the order the classifier uses for its columns. Column 0 corresponds to the first entry of classes_, column 1 to the second, and so on. Without this lookup, an argmax result tells you a position but not which class that position stands for.

The course example uses the iris dataset, where the targets are given as class names. The classifier is fitted on those names, so classes_ holds the names in sorted order. The argmax of the scores produces positions, and indexing classes_ with those positions reproduces the labels that predict returns.

decision_function is a second method that many classifiers provide. It returns decision values rather than probabilities. In the multiclass case its shape is (n_samples, n_classes), the same layout as predict_proba, and the argmax across columns recovers the predicted class in the same way. The numbers are scores, so you should not read them as probabilities that sum to 1.

Binary classification is the special case. With two classes, decision_function returns a single value per sample, so its shape is (n_samples,) with no second column. That one value refers to the positive class, which is classes_[1]. The course notes that this layout is mostly a matter of historical convention rather than a different mechanism.

String labels and integer labels that do not start at 0 are where index-based shortcuts go wrong. The argmax always returns a position between 0 and K minus 1. It does not return a label. If you treat that position as if it were the label, the answer is correct only in the lucky case where the labels happen to be exactly 0, 1, 2 in the same order as the columns.

Work through this step on your own before you read the answer. A model has classes_ equal to ['cat', 'dog', 'bird'] and returns the probability row [0.15, 0.30, 0.55] for one sample. Which label does the model predict, and what would a shortcut that uses the argmax position alone report instead?