Logits and Model Output

A language model does not finish its work when it produces its raw output. That output is a list of numbers, one for each token in the model's vocabulary. It is tempting to read those numbers as the model's confidence about each token. Many learners assume that a bigger number means a more likely token, and that the numbers are already percentages that add up to a whole. This article tests that belief, shows where it breaks, and then shows the standard tool used to turn raw output into something that can be treated as a probability.

The belief usually goes like this: the model outputs a vector of scores, the largest score wins, and so the scores must be the model's probability estimates. The first half is partly true. Larger logits do correspond to higher probabilities. The second half is where the reasoning goes wrong. A larger logit ranks a token higher, but the logit itself is not a probability. The scores are raw outputs, and they have not yet been arranged into a form that probability rules require.

Think of the raw output as a vector with one entry per token. Each entry is called a logit. The vector might look like [2, 1, 0] for a three-token vocabulary, or [-1, -2] for a two-token one. Each position stands for one token, and the value at that position is the model's raw score for that token. Nothing in the vector guarantees that the entries are non-negative or that they add up to one.

The source lists three properties that separate logits from probabilities. Each one is a property a probability must have, and logits do not automatically have it. Logits do not represent probabilities. They do not add up to one. And they can be negative, whereas probabilities have to be non-negative. Because of these three gaps, a raw logit vector cannot be used directly as a probability distribution, even though its largest entry still points at the most favored token.

A softmax layer is the component that converts a logit vector into probabilities. Suppose the model's vocabulary has N tokens, and the logit vector is written as x_1, x_2, up to x_N. The probability assigned to token i is given by the formula below, where the sum in the denominator runs over every token j from 1 to N. Read the formula as a recipe: take each logit, transform it with the exponential, and divide by the total of all the transformed values.

Notice what the formula guarantees. Each exponential is a positive number, so every numerator is positive, and so is the denominator. Each probability is therefore non-negative. The denominator is exactly the sum of all the numerators, so when you add the probabilities for every token, the numerators cancel the denominator and the total is one. These are the two properties that the raw logits lacked, and the formula produces them by construction.

A common surprise is that logits can be negative, while probabilities cannot. Softmax resolves this without any special rule. Because the exponential of any real number is positive, a negative logit becomes a small positive number in the numerator. A logit that is negative therefore receives a small but nonzero probability, and the normalization step still makes the total equal one.

Once a softmax layer has produced a probability for every token, the model still has to choose which token to output. This choice is where sampling strategies operate. The source explains that the right sampling strategy can make a model's responses better suited to a given application. One strategy can make responses more creative, while another can make them more predictable. Many strategies have been introduced to nudge the model toward responses with specific attributes, and you can also design your own, though doing so typically requires access to the model's logits.

The flowchart makes the ordering clear. Sampling strategies do not operate on raw logits in the ordinary description; they receive the probabilities that softmax has already produced. That is why the probability distribution has to be valid first: a strategy that picks among options weighted by these values relies on them being non-negative and summing to one.