How AI Models Generate Text Token by Token

Suppose you ask a friend, a minute apart, which cuisine is the best in the world. Your friend will almost certainly give you the same answer both times. Now suppose you put the same question to an AI model twice and get two different answers. That gap between human consistency and machine variation is the subject of this article.

The goal is to understand where the variation comes from, why it is not a bug in the usual sense, and what it means for anyone building or relying on AI tools. The answer begins with a single idea: the model does not look up one fixed answer. It draws its response from a set of possibilities.

AI models are probabilistic because of the way they sample their responses. Rather than locking in a single answer, the model holds a set of possible continuations, each with a probability attached, and selects one of them. The selection is a draw, not a lookup.

A lookup returns the same stored entry every time you ask for the same key. A draw works differently. Imagine a bag containing weighted tickets, where some tickets are more likely to be pulled out than others. Each time you reach in, you might get a different ticket, even though the bag itself never changes. Once you see the output as the result of a draw, the variation between runs stops looking mysterious.

Generation happens in small steps. At each step the model works with candidate pieces of text that could come next, which this article calls tokens, and it assigns each candidate a probability. Together those probabilities form a distribution over the candidates. The probabilities must add up to a total of one, because exactly one candidate is eventually chosen.

The figure below shows one such step after the prompt The capital of France is. The numbers are illustrative, chosen for this lesson rather than taken from a real model, but they follow the structure the source describes: each candidate has a probability, and one candidate is drawn in proportion to its probability.

A response is not produced in a single lookup. It grows through a repeated cycle. The model receives the prompt, assigns probabilities to the possible next pieces, draws one, and appends it to the response. Then the cycle repeats using the text as it now stands, with the newly added piece included in what the model sees.

Because each draw changes the text, every later step is conditioned on the choices made earlier. This is why one early draw can change everything after it. The loop has no step that goes back and re-checks an earlier choice against the world.

A simple deterministic example is a rule that adds two numbers. Given 2 and 3, the rule always returns 5, no matter how many times you run it or on which day. A probabilistic example is a weighted spinner that lands on red with probability 0.9 and on blue with probability 0.1. Each spin is a fresh draw, so two spins with identical settings can land on different colors.

The model's generation is probabilistic in the second sense. Its probabilities are fixed for a given prompt, but the final choice is a draw from those probabilities, so the outcome can vary even though nothing about the setup has changed.

Consider a prompt that asks the model to name a fruit. On the first run, the first draw lands on apple, and the continuation builds from there. On the second run, the first draw lands on mango, and the continuation builds from that different starting point. Neither run is wrong, and neither was guaranteed. The divergence happened at the first draw, and every later token followed from that early choice.

The two runs share the same prompt and the same probabilities. They differ only because the first draw landed on a different candidate. The later tokens are not separately random in an arbitrary way; they are drawn from distributions that now depend on the text already produced.

Inconsistency means a model generates very different responses for the same prompt or for slightly different prompts. Because each response is a draw from a distribution, two runs can land on different parts of that distribution. The variation is not an accident layered on top of the model. It is what sampling produces.

A small change in wording can also shift the probabilities. Rephrasing a question changes the text the model conditions on, which changes the weights on the candidates. A rephrased question may therefore lead the model to a different region of possible answers, even when a human would consider the two questions identical in meaning.

A hallucination is a response that is not grounded in facts. The same sampling mechanism that gives a model its flexibility can also let it produce false content. The model does not check its output against the world. It produces continuations that its training data makes probable, and any continuation with a nonzero probability can, in principle, be drawn, however unlikely or wrong it is.

Consider how this can happen. Suppose an essay claiming that all US presidents are aliens was published online and included in the training data. The model may then assign that claim some probability, and sometimes it will output that the current US president is an alien. To a reader who knows this is untrue, the model appears to be inventing it. In fact the model is reflecting what its data contained, filtered through probability.

The same property that causes trouble in factual settings is useful elsewhere. Creative work often means exploring beyond the most common paths and thinking outside the box. A model that samples from a wide set of possibilities can act as a sidekick to a creative professional, brainstorming many ideas quickly and proposing designs that have never been seen before.

The source is explicit about the tradeoff. The probabilistic nature that helps with creativity can be a real problem for everything else. This is why engineers building AI applications have to decide where variability is welcome and where it must be constrained.

Use this small distribution for the prompt Name a fruit. The candidate next tokens are apple with probability 0.40, mango with probability 0.25, banana with probability 0.20, and cherry with probability 0.15. The probabilities add to 1, so the distribution is valid.