Vocabularies and Token Probabilities

A language model does not finish its work when it produces its raw output. That output is a list of numbers, one for each token in the model's vocabulary. The vocabulary is the full set of tokens the model can produce, and the raw output assigns one number to every entry in that set. It is tempting to read those numbers as the model's confidence about each token. The larger the number, the more the model seems to favor that token, and the numbers look like they might already be percentages that add up to a whole.

This article tests that belief, shows exactly where it breaks, and then introduces the standard tool used to turn raw output into something that can be treated as a probability. Keep the belief in mind as you read: the model outputs a vector of scores, the largest score wins, and so the scores must be the model's probability estimates. Part of this is true, and part of it is where the reasoning goes wrong.

The first half of the common belief is partly true. A larger logit does rank a token higher, so the largest entry still points at the most favored token. The second half is where the reasoning goes wrong. The scores are raw outputs, and they have not yet been arranged into a form that probability rules require. The source identifies three properties that separate logits from probabilities. Each one is a property a probability must have, and logits do not automatically have it.

Because of these three gaps, a raw logit vector cannot be used directly as a probability distribution. A probability distribution over the vocabulary needs every entry to be non-negative and the entries to sum to one. Raw logits meet neither requirement by default, even though their ordering is still informative.

To see the mapping concretely, imagine a tiny vocabulary of three tokens: cat, dog, and fish. The model produces the logit vector [2, 1, 0], meaning cat has logit 2, dog has logit 1, and fish has logit 0. Each token keeps its own logit, and after conversion each token also gets its own probability. The sequence of positions in the vector stays fixed, so each token's logit and probability can be traced across the whole vector.

Notice that the ordering survives the conversion. Cat has the largest logit and also the largest probability, which matches the source's statement that larger logits correspond to higher probabilities. What changes is the form of the numbers. The logits are 2, 1, and 0 with a total of 3, while the probabilities are about 0.665, 0.245, and 0.090 with a total of exactly one.

A softmax layer is the component that converts a logit vector into probabilities. Suppose the model's vocabulary has N tokens, and the logit vector is written as x_1, x_2, up to x_N. The probability assigned to token i is given by the formula below, where the sum in the denominator runs over every token j from 1 to N.

Read the formula as a recipe. Take each logit x_i, apply the exponential to get $e^{x_i}$, and then divide by the total of all the transformed values. The numerator belongs to one token, while the denominator is shared by every token in the vocabulary.

Now apply the recipe to the logit vector [2, 1, 0] from the opening. The steps are the same for any vector, so this one works as a template for the process.

A common surprise is that logits can be negative, while probabilities cannot. Softmax resolves this without any special rule. The exponential of any real number is positive, so a negative logit becomes a small positive number in the numerator. A negative logit therefore receives a small but nonzero probability, and the normalization step still makes the total equal one.

Notice the relative sizes. The logit -1 is larger than -2, and after exponentiation its value 0.368 is larger than 0.135. Exponentiation preserves ordering, so the larger logit still corresponds to the larger probability even when every logit is below zero.

Once a softmax layer has produced a probability for every token, the model still has to choose which token to output. This choice is where sampling strategies operate. The softmax layer's job ends when the distribution exists. Sampling begins with that distribution as its input.

The source explains that the right sampling strategy can make a model's responses better suited to a given application. One strategy can make the model's responses more creative, while another can make them more predictable. Many strategies have been introduced to nudge the model toward responses with specific attributes. You can also design your own, although the source notes that doing so typically requires access to the model's logits.