Instruction Hierarchy in Foundation Models

When people first build on a language model, they often assume that the message format is a detail the model will sort out on its own. The reasoning goes like this: you have some instructions, you have a question, and as long as both reach the model, the model will work out which part is which. Under that belief, the exact spacing, line breaks, and special markers around the text look like cosmetic choices.

The source material shows why that belief is risky. Template mismatches often fail silently, and even a small formatting difference, such as an extra new line, can change the model's behavior. Nothing crashes and no error message appears. The model simply receives a slightly different sequence of characters and responds differently. This article follows one request from the moment its pieces are written down to the moment the model reads them, so you can see exactly where that sequence is built and where it can go wrong.

A foundation model that accepts a system prompt and a user prompt does not read them as two separate objects. The source describes the model combining the two prompts into a single prompt, typically by following a template. This step happens before the model does any work, which is why it is easy to overlook. It determines the exact sequence of characters the model actually reads.

A prompt template is a fixed layout with empty slots. The slots are filled with the content of a particular request, and the result is one complete piece of text. The template decides where each piece goes, which markers separate them, and what text surrounds them. The model then receives the combined result as its single input. The diagram below traces that path from the two prompts to the one input.

The word template is used for two related but distinct things, and confusing them is a common source of errors. The first is the chat template that a model's developers define for their model. It describes how that model expects its input to be structured, and it can usually be found in the model's documentation. The second is the prompt template an application developer writes to fill in their own prompts with specific data, such as a product name or a document. Any application developer can define the second kind. Only the model's developers define the first.

The source uses Llama 2's chat model as its first example. Its template places the system prompt inside special bracket markers and then places the user message after them, followed by a closing marker. Read from left to right, the template starts with a begin marker, opens an instruction block, opens a system block, inserts the system prompt, closes the system block, inserts the user message, and closes the instruction block. The diagram below labels each part in that order.

Two of these parts are fixed and two are slots. The markers are the same for every request, and the system prompt and user message are the only parts that change. This is what makes a template a fixed layout with empty slots: the layout never moves, and only the contents of the slots differ between requests.

Templates are not permanent. For the Llama 3 chat model, the provider changed the layout. The same underlying idea, a system part followed by a user part, now looks quite different. The new layout uses header markers that name each role, end-of-turn markers that close each part, and a final header that prompts the model to begin its reply. A string that was correct for one version may be wrong for the other, even though the content is identical.

The markers themselves are not ordinary words. The source explains that each span between a less-than-bar delimiter and a bar-greater-than delimiter is treated as a single token by the model. A marker such as the start-header marker is therefore handled as one unit, not as a sequence of letters that might be rewritten or split by accident. Changing even a small part of one of these markers produces a different input from the one the model was trained to expect.

Model providers often say that well-crafted system prompts improve results. Anthropic's documentation, for example, says that assigning a role or personality through a system prompt helps the model stay in character more effectively across a conversation. That sounds as though system prompts must be handled in a special way. The source explains that this is not the mechanism. Under the hood, the system prompt and the user prompt are concatenated into one final prompt before the model sees them. From the model's side, both are processed the same way.

If the system prompt is processed no differently, the apparent boost must come from something else, and the source offers two likely explanations. The first is position: the system prompt sits at the front of the final prompt, and a model may simply handle instructions that appear first more reliably. The second is training: a model may have been post-trained to pay closer attention to system prompts. The source points to work on an instruction hierarchy, which trains models to prioritize privileged instructions, and notes that this training also helps defend against prompt attacks.

When a model's output seems wrong, it is tempting to blame the model or the wording of the question. The source's guidance points somewhere more specific. Because the failure is silent, the most useful place to look is the step where the prompt is assembled, and the exact text that results from it. The pipeline below shows the order of steps between the application's code and the model. The highlighted step is the one whose output you should inspect first.

The final string is the most reliable place to inspect, because it is the only version that matches what the model actually reads. Earlier stages can look correct while the final string is still malformed, for example when a layout was built against the wrong version of a template. Comparing the printed final text with the template documentation is a simple habit that catches many of these problems.

The consequences of a mismatch are easy to underestimate. The source reports that using the wrong template can produce puzzling performance problems. The mistakes below are the ones most worth watching for.

Because the failure is silent, the practical defense is to inspect the exact text that will reach the model. The checks below follow the order in which the source's habits can be applied. Run them on the assembled final string, not on the pieces you started with.