Providing Relevant Context to a Model

Foundation models can do a remarkable range of things, but they only do what you actually ask for. Typing a request into a model takes seconds, and most people who first try one get a useful reply right away. That experience leads many people to conclude that writing instructions is simply a matter of being understandable. This article examines why that conclusion is incomplete and what it takes to communicate with a model so that it reliably produces the result you want.

Before reading on, commit to an answer. Suppose you change one word in an instruction that otherwise stays the same. Would you expect the model's response to change only slightly, or could the difference be large? Keep your answer in mind, because the rest of the article explains what determines the answer.

The source frames this work as human-AI communication. Anyone can communicate, but not everyone communicates well. Prompt engineering sits at the center of that communication, because the instruction is the only channel through which you tell the model what the task is.

The most common misconception is that prompt engineering is easy to do well because it is easy to begin. Starting is indeed effortless, and that ease misleads many people into thinking that doing it well is equally effortless. These are two different skills. A first attempt that produces something plausible tells you very little about whether the instruction is clear, complete, and robust enough for the task.

Follow one task through two versions of the prompt. The task stays the same in both versions: a short summary of a meeting transcript. Only the precision of the instruction changes. This is the mechanism to watch: the model receives only the text you send, and whatever the text leaves unspecified, the model has to guess.

The gap between a first draft and a refined prompt grows as the task becomes more demanding. A short, familiar task may succeed with rough wording. A task with several constraints, a specific audience, and a required structure leaves far more room for the model to go wrong, so the same care in wording matters more.

The gap between starting and doing well comes down to sensitivity. Sensitivity describes how much a model's response changes when the prompt changes. The amount of crafting a task needs depends on this property: if a small change can cause a big change, more crafting is necessary.

A model that responds nearly the same way to many different phrasings is forgiving. Rough instructions may still work, and little crafting is needed. A model that responds very differently to small wording changes is unforgiving. The same task may succeed or fail depending on details that look trivial to a human reader.

Sensitivity varies from model to model and from task to task, so no single phrasing rule guarantees success. The practical consequence is that you must test how a prompt behaves rather than trust that a sensible-sounding instruction will work.

The source identifies three ingredients that are essential for clear communication: clear instructions, examples, and relevant information. The same is true whether you are writing to an AI or to another person. Each ingredient does a different job, and all three serve one single task.

A model that has only the instruction must guess at the rest. Each of the three subsections below describes one ingredient and what goes wrong when it is missing.

The source notes that small techniques can produce noticeable gains. One example it gives is asking the model to slow down and think step by step. Such a simple request changes how the work is organized, and it can yield surprisingly large improvements in the response.

The same ability that makes foundation models useful also creates a risk. Models are useful because they can follow instructions, but that ability opens them up to prompt attacks, in which bad actors get a model to follow malicious instructions. The source describes security as an ever-evolving cat-and-mouse game.

Chat models expect conversations to be formatted in a particular way, using a chat template. The template defines how the system, user, and assistant messages are laid out in the final text the model actually reads. The source reports that deviations from the expected template usually cause model performance to degrade.

In the strip above, a mismatch changes one or more of the boundary markers, so the model may read where one message ends and the next begins differently from what the developer intended. The source notes that a mismatch is not always an obvious failure: although uncommon, a deviation can sometimes make the model perform better.

Because sensitivity varies, the only reliable check is to observe the behavior. Choose a task you already understand, such as summarizing a short paragraph for a specific reader. Write a rough version of the instruction and a refined version that adds an audience, a format, and a length limit. Run both on the same input, then vary one word in each instruction and observe whether the output shifts.