Softmax and Probability Scores
The probability scores are then passed to argmax, which returns the position of the largest value.
Why a Classifier Returns Numbers, Not Words
A text classifier does not hand you the word spam or not spam directly. What it hands you is a set of numbers, and those numbers have to be translated into a label before anyone can count how often the model was right. This article follows one example through that translation.
The output we follow is the output for the last token of an input text. In this setup, the last token's output is the one used to make the classification decision. Its raw output is a row of values, and the job of the conversion is to decide which position in that row wins.
What do you think happens?
The last token's raw values are -3.5983 at position 0 and 3.9902 at position 1. Argmax returns the position of the largest value. Which position will it return, and what would that position mean for the input text?
Reveal answer
Position 1 is returned, because 3.9902 is larger than -3.5983. In this setup, index 1 is the label for spam, so the input text is classified as spam.
Argmax does not return the largest value itself. It returns the position of that value, and that position is then read as a class. Here the position is 1, and the source's mapping assigns index 1 to spam.
Tracing One Output Row Through the Conversion
The source first prints the raw output for the last token and then converts that output into probability scores using softmax. The probability scores are then passed to argmax, which returns the position of the largest value. Each stage takes the row produced by the previous one, so the same two positions travel through the whole sequence.
Read the diagram from top to bottom. The two raw values sit in the row at positions 0 and 1. Argmax compares them and selects the position holding the largest value, which is position 1. That selected position is then read as the class, which here is spam. The key question to keep in mind is simply which position holds the largest value and what that position stands for.
What Argmax Returns
Argmax converts a row of model output into a class label by returning the position of the largest value. The value of the largest entry is not what gets returned; the position is.
Reading a two-value output row
The last token's raw output row is -3.5983 at position 0 and 3.9902 at position 1. Find the predicted class label using argmax.
- 1List the values and their positions
Position 0 holds -3.5983 and position 1 holds 3.9902.
- 2Find the largest value
3.9902 is greater than -3.5983, so the largest value is 3.9902.
- 3Return its position
The largest value sits at position 1, so argmax returns 1.
- 4Map the position to a class
In this setup, index 1 is the label for spam, so the predicted label is spam.
Argmax returns index 1, and the predicted label is spam.
Where Softmax Fits and Why It Is Optional
Softmax converts raw output values into probability scores. The source points out that using softmax in this step is optional. The reason is that the largest raw output already marks the highest probability score, so softmax cannot change which position wins. Softmax rescales the values into probabilities, but it preserves their ordering, so the position of the largest value stays the same.
| Path | Values argmax sees | Largest position | Predicted label |
|---|---|---|---|
| Raw values, softmax skipped | -3.5983 and 3.9902 | 1 | spam |
| Softmax applied first | about 0.0005 and about 0.9995 | 1 | spam |
Both paths put the largest value at the same position, so the predicted label is unchanged.
The softmax values in the table are approximate, computed from the raw pair. Softmax makes the winning probability close to 1 and the losing one close to 0, but the question argmax asks, which position is largest, has the same answer on either path. That is why skipping softmax leaves the predicted label unchanged.
Mistakes When Reading Model Output
Common mistake Returning the largest value instead of its position
What it looks likeReading the row -3.5983, 3.9902 and reporting 3.9902 as the prediction.
Why it failsArgmax returns the position of the largest value, not the value itself.
Do this insteadReport index 1, then map that index to its class label, spam.
Common mistake Thinking softmax is required before argmax
What it looks likeAssuming the row must be converted to probabilities before a label can be chosen.
Why it failsThe largest raw output already marks the highest probability, so softmax does not change which position wins.
Do this insteadTreat softmax as optional for the label, and apply it only when you need probability values.
Common mistake Forgetting the position-to-label mapping
What it looks likeStopping at index 1 and treating it as the answer.
Why it failsA position is only a number until it is translated into a class.
Do this insteadUse the mapping for your setup; in this example, index 1 is the label for spam.
From One Label to Classification Accuracy
The same argmax conversion becomes a measurement tool once you apply it to many examples. Classification accuracy is the percentage of correct predictions across a dataset. To compute it, you run the argmax-based prediction on every example, compare each predicted label with the correct label, and calculate the proportion that match.
The source packages this procedure in a function named calc_accuracy_loader, which is the function listed in the accuracy listing. Its idea matches the single-example case: each example produces one predicted label, and accuracy is the fraction of those predicted labels that are correct. Each example is treated the same way, so a single pipeline serves the whole dataset.
Practice: Predict the Label Yourself
A last-token output row contains 2.10 at position 0 and -0.75 at position 1. Using the mapping from this article, where index 1 is spam, predict the argmax position and the predicted label. Then say whether applying softmax first would change your answer.
Hints
- Compare the two values and decide which is larger before anything else.
- The answer is a position, and then you map that position to a class.
- Softmax preserves the ordering of the values, so think about whether the largest position can change.
Key Takeaways
- Argmax converts a row of model output into a class label by returning the position of the largest value, not the value itself.
- In the source's example, the raw pair -3.5983 and 3.9902 yields index 1, which is the label for spam.
- Softmax is optional for choosing the label, because the largest raw output already marks the highest probability and softmax cannot change which position wins.
- Classification accuracy applies the same argmax prediction to every example in a dataset and reports the percentage of predicted labels that are correct.