Persisting Information and Model Consistency

Imagine someone asks you to rate the same joke on a scale from 1 to 5, and then asks again a few minutes later. Would you give the same number both times? Many people find that harder than it sounds. A rating like this is subjective: there is no answer key to check against, so the number depends on how the rater feels in the moment and on what they remember about their own earlier judgment.

This small scenario captures the topic of this article, which is model consistency. An AI model is consistent when it gives steady answers to the same question or to similar questions. The source's answer to how a model can be helped to stay consistent is surprisingly human: it depends on the model being able to look back at what it already said.

The easiest way to see the mechanism is to follow the messages in order. In the diagram below, the user asks for a rating, the model answers, and then the same question comes back. The key question is whether the earlier answer is available when the second answer is produced. The source's point is that a model which can reference its previous answers has something to calibrate against.

Read the diagram as a conversation rather than as a single answer. Step 2 is the step that matters most for consistency. Rating 1 is not just a number that disappears once it is spoken; it is part of what the model can see when it reaches step 4. The diagram is a teaching device built from the source's rating scenario. The source describes the principle at the level of the idea and does not walk through a specific session.

Consistency is about whether the answers agree with each other across repeated questions. The source places its discussion mainly on subjective questions, such as rating a joke from 1 to 5. These are the questions where consistency is most at risk, because a rating like this has no single correct value. That is exactly why repeated answers can drift.

The central idea in the source is that a model which can reference its previous answers can calibrate future answers to match them. The source compares this to people. When a person is asked a subjective question twice and remembers the first answer, the earlier response acts as an anchor. It gives the second answer something to be compared against, so the rating does not drift without reason.

Notice what this mechanism depends on. The model must be able to reference its previous answers. If earlier answers are not available when the model responds to a later question, it has nothing to calibrate against. The source's point is therefore conditional: the ability to look back is what makes improved consistency possible. The comparison below contrasts the two situations.

A repeated subjective task can be read as a loop. In each round the model looks at its prior answers, compares them with the new question, and then chooses its next rating. The loop below shows the same cycle repeating for each new question.