Decision Function Scores in Classification

predict_proba returns one row per sample, and for binary classification each row has exactly two entries, giving the shape (n_samples, 2).

Programming

Predicting Before Reading the Output

Many classifiers offer a method called predict_proba. Instead of returning one label per sample, it returns a number for each class that says how likely that class is, according to the model. In other words, the method is making a promise about each sample, and this article is about reading that promise correctly. Before going further, it helps to make a prediction of your own about what the method's output looks like.

What do you think happens?

A binary classifier is given 6 test rows and asked for predict_proba. What shape does the returned array have?

(6,)
(2, 6)
(6, 2)
(6, 6)

Reveal answer

Answer:

(6, 2)

One row is returned per sample, so there are 6 rows. For binary classification each row has exactly two entries, one for each class, so the second dimension is 2.

The Shape Pattern: Samples by Classes

The shape pattern is always (n_samples, 2) for binary classification. The first number is the number of samples, so each sample gets its own row. The second number is 2 because a binary problem has exactly two classes, and the method gives one probability for each of them. If your data had 3 classes, the same idea would suggest a different second dimension, but this article stays with the two-class case.

How do the rows and columns of the returned array line up with the samples and the two classes? With 3 samples, the array has 3 rows and 2 columns, so each row holds one sample's pair of probabilities.

Read the grid row by row. Each row is one sample, and the two columns inside that row are the two classes. Adding more samples adds more rows but never changes the number of columns when the problem is binary. The first dimension grows with your data; the second stays fixed at 2.

Reading One Row Entry by Entry

Each row is a pair of numbers. The first entry is the estimated probability of the first class, and the second entry is the estimated probability of the second class. Because each value is a probability, every entry lies between 0 and 1. Because the two classes cover all the possibilities for a sample, the two entries in a row always add up to 1.

Which position in a row corresponds to which class? Position 0 holds the probability of the first class, and position 1 holds the probability of the second class.

The position of an entry is the only thing that tells you which class it belongs to. The array does not print class names next to the numbers, so your code has to keep track of which column is which class. When you report results, state which column corresponds to which class so that readers do not reverse them. In scikit-learn, the class labels the model learned are stored in the classes_ attribute, and that is the place to confirm the column order rather than assuming it.

Worked example

Reading the first row of a result

The first row of predict_proba output is [0.8, 0.2]. What does this row say about the sample?

1
Identify the entries
The row has two entries, as expected for binary classification.
2
Map position to class
Entry 0 is the estimated probability of the first class, which is 0.8.
3
Map the second position
Entry 1 is the estimated probability of the second class, which is 0.2.
4
Check the range
Both values lie between 0 and 1.
5
Check the sum
0.8 + 0.2 = 1, so the row is consistent with being a valid pair of probabilities.

Result

The model estimates an 80 percent chance that this sample belongs to the first class and a 20 percent chance that it belongs to the second class.

Checking That a Row Is Valid

The source gives a quick test for a valid output: every probability lies between 0 and 1, and the two entries in a row sum to 1. For a binary row with entries $p_{0}$ and $p_{1}$, the check is written as the two conditions below.

Formula

Row	Values	Between 0 and 1?	Sums to 1?	Verdict
Valid	[0.8, 0.2]	Yes	Yes	Consistent with a valid output
Valid	[0.35, 0.65]	Yes	Yes	Consistent with a valid output
Invalid	[0.7, 0.4]	Yes	No, sums to 1.1	Something upstream is wrong
Invalid	[1.2, -0.2]	No	Yes, sums to 1	Values fall outside the allowed range

Each invalid row fails a different check, so both checks are needed.

Tracing a Short Program

A common habit is to pass only a slice of the test data to predict_proba, so the output is short enough to read. Passing the first six test rows returns six pairs of probabilities, and the rows still follow the same rules: each entry lies between 0 and 1, and each pair sums to 1. The program below does exactly that, and it prints the three things you should check in order: the shape, the sums, and one row.

python

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=100, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

model = LogisticRegression()
model.fit(X_train, y_train)

probs = model.predict_proba(X_test[:6])
print(probs.shape)
print(probs.sum(axis=1))
print(probs[0])

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=100, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

model = LogisticRegression()
model.fit(X_train, y_train)

probs = model.predict_proba(X_test[:6])
print(probs.shape)
print(probs.sum(axis=1))
print(probs[0])

Output (expected)

(6, 2)

Notes on the code

make_classification creates a two-class dataset with 100 samples
train_test_split holds back a test set that the model has not seen during fitting
predict_proba is called on only the first six test rows
probs.shape should be (6, 2): six rows, two columns
probs.sum(axis=1) adds the entries of each row; each value should be very close to 1
probs[0] selects the first row, a pair of probabilities for one sample

Code walkthrough

This program follows the same order as a careful reader should use. It builds the data, splits off a test set, fits a model, and then calls predict_proba on six rows. The printed shape comes first because a wrong shape means every later number is being read from the wrong place. The row sums come next to confirm the outputs are valid probabilities. The single row at the end is read entry by entry, with position 0 as the first class and position 1 as the second class.

1
Lines 1 to 3 import the tools for generating data, fitting a classifier, and splitting the data.
2
Line 5 creates 100 samples with 4 features each, and their binary labels in y.
3
Line 6 splits the data so that the model is evaluated on rows it did not learn from.
4
Lines 8 and 9 create the classifier and fit it on the training rows only.
5
Line 11 calls predict_proba on the first six test rows; this is the call where the output shape is decided.
6
Line 12 prints the shape, which should read (6, 2) for six samples in a binary problem.
7
Line 13 sums each row; values near 1 confirm the rows are valid pairs.
8
Line 14 prints a single row so you can read which entry is which class.

At which step does the wrong value or shape first appear? Check the steps in order and stop at the first one that does not match your expectation.

Mistakes Beginners Make With Output Columns

Common mistake
Assuming the shape is (2, n_samples) or (n_samples,)
What it looks like
Expecting probs.shape to read (6,) for six test rows and then indexing probs by sample first.
Why it fails
The first dimension is the number of samples, and each sample gets its own row, with two entries per row.
Do this instead
Print the shape right after the call and confirm it matches (n_samples, 2).
Common mistake
Reading the wrong column for the class of interest
What it looks like
A program meant to report confidence in the second class reads probs[:, 0] instead of probs[:, 1].
Why it fails
Column 0 holds the probability of the first class, so the reported number belongs to a different class.
Do this instead
Decide which column is the class of interest, state it in the code, and check it against the model's classes_ attribute.
Common mistake
Treating a row whose sum is not 1 as a valid probability pair
What it looks like
Accepting [0.7, 0.4] because both numbers look like plausible probabilities.
Why it fails
The two entries of a binary row must add up to 1 because the two classes cover all possibilities for a sample.
Do this instead
Sum each row and check that every value lies between 0 and 1; investigate any row that fails either check.
Common mistake
Reporting results without naming which column is which class, so readers reverse the two numbers

Practice: Find the First Wrong Step

MEDIUM

A colleague says their model's predict_proba output is wrong. They fitted the model, called predict_proba on the first six test rows, and printed the result. The printed shape is (6, 3), and the program then reads probs[:, 1] to report confidence in the second class. Working through the program in order, at which step does the unexpected value first appear, and what would you check to confirm it?

Hints

Start with the shape. The second dimension should equal the number of classes, and this problem is described as binary.
Compare the printed shape with the expected (n_samples, 2) pattern before looking at any individual values.
If the shape is already wrong, later steps such as reading a column are reading from a result that does not match the binary pattern.

Your attempt (not saved)

EASY

Write down the shape you expect from predict_proba for 10 samples in a binary problem, then write one row that is valid and one row that is not, explaining which check each fails.