Writing Good Instructions for Models

Imagine two assistants receive the same question about a printer's speed. The first assistant sees only the question. The second assistant also sees the printer's technical specification sheet. Which one is more likely to give a correct answer, and which one is more likely to invent a number that sounds plausible? The difference lies in the context each assistant was handed, and that difference is the starting point for understanding security risk in AI systems.

This article follows three questions that the source treats as the core of security in foundation model systems: what information the model is shown, where that information came from, and what the model is allowed to do after it reads it. Security concerns do not come only from the model's wording. The sections below trace how context reaches a model, how tools widen what it can do, and how scoping that context keeps the risk contained.

A model needs two kinds of material to complete a task. The first is instructions, which describe how the task should be done. The second is context, which is the actual information needed to handle a particular request. For a given application, the instructions tend to be the same for every user and every question, while the context changes with each query.

Context is where most of the variety, and most of the risk, lives. When context is missing, the model is more likely to make mistakes and to produce confident statements that are not grounded in anything it was given. People behave the same way: someone asked a question without the relevant facts is more likely to guess wrongly. Careful context construction, meaning the deliberate selection of what goes into the prompt for each query, is therefore both a quality practice and a safety practice.

Retrieval-augmented generation, usually called RAG, is built on a two-stage flow. A retriever searches a memory store and selects a small set of items that appear relevant to the query. Those items are then placed into the model's input, and the model writes its response while reading them. The model does not have to memorize every fact in advance, and the store can be updated without retraining the model.

Reading the diagram from left to right: the query enters the retriever, which searches the memory store and keeps only the items it judges most relevant. Those selected items are placed into the model input next to the question. The model never sees the whole store; it sees only what was selected for this query, and it writes its answer from that material.

The retrieve-then-generate idea predates the modern name. An early system first pulled the five Wikipedia pages most relevant to a question and then had a reader model generate an answer from them. The term retrieval-augmented generation was later introduced for a broader setting, where the full body of knowledge is too large to place inside the model's input at once. In that setting, only the information the retriever judges most relevant is passed along.

A common hope is that models with very long context windows will make retrieval unnecessary, since everything could simply be placed in the prompt. The source argues against that view on two grounds, and both explain why selecting context for each query remains useful.

The source also points to practical testing of how well models find specific information inside large inputs. It cites a 2024 writeup in which a practitioner tested a model on information embedded in long material about doctor visits. The lesson for security work is that a model's use of a long context should be tested, not assumed, before anyone relies on it for sensitive decisions.

The agentic pattern reaches the same goal of better context by a different route. Instead of retrieving from a fixed store, the model is allowed to call tools such as web search or news APIs to gather information it needs. The model decides which tool to use, the tool returns results, and those results become part of the context the model uses to continue.

Follow the loop in the diagram. The query starts the process, and the model decides at the decision node whether it needs a tool. If it does, the tool is called and its result returns, is added to the context, and the flow loops back to the same decision. If the model no longer needs a tool, it stops looping and gives its final answer. Each pass around the loop makes the context larger and more tailored to the task.

Tools do more than supply information. They are the means by which a model can act on the world, and this is what makes the pattern powerful. External tools can compensate for a model's shortcomings and widen what it can do, which is why the agentic pattern is considered capable of automating many everyday tasks. The same reach that makes agents useful is what makes their security posture deserve close attention.

Each tool connection is a path through which information comes in and, for some tools, through which actions can go out. Every such path is a place where a security review should ask what the model is permitted to request and what it is permitted to do with the result. Retrieved text is also an input path: any text that reaches the prompt can influence what the model produces.

The concept map shows where the risk can enter. Retrieved text and tool results both shape the model output. From there, the output can be displayed safely or it can be handed onward, either to an interpreter or as a request that triggers a tool action. The safest default is to treat model output as untrusted data.

One concrete risk named in the source is remote code execution. The source reports that such a risk was found in LangChain in 2023 and points to public GitHub issues describing it. The underlying danger is easy to state: if text produced by a model, or text that a model was influenced by, is passed to something that executes it as code, then the model's output can determine what runs on a machine.

The flowchart idea behind this risk is a decision node. Whether the system displays a model's output or hands it to an interpreter is a choice the designer makes, and that choice determines whether an unexpected output can cause an unexpected action. The source treats this as a general model of control flow rather than a description of one specific issue.

A useful way to think about RAG is as a method of building a different context for each query, rather than reusing one large, fixed context everywhere. That design has a direct security benefit. Private data about one user can be retrieved only for queries that belong to that user, so it never enters the prompt for someone else's question.

Scoping shrinks the reachable surface in two ways. Narrow retrieval means that a query can only pull in data that belongs to the person asking, so a flaw or an unexpected output cannot expose other users' material. Narrow tool access means that each tool connection is reviewed for what the model may request and what it may do with the result, so the set of possible actions is limited to what the task needs.

Several errors recur when teams build systems on retrieval, tools, and long prompts. Each one comes from treating the model as the place where security decisions get made, when the surrounding system has to make them.