Reducing Hallucinations in Model Outputs
Sufficient context means the information a model needs is present in what it receives, which helps responses and reduces the chance of hallucination when that information is missing.
Why Better Wording Often Is Not Enough
When a model gives a disappointing answer, the usual reaction is to rewrite the instructions, add more emphasis, and hope the model behaves better. That habit rests on a belief: that the way you phrase a request is the main lever you have over what the model knows and says. This article tests that belief against a more basic question. What information does the model have access to when it writes its answer, and what should it have access to?
The answer shapes everything that follows. A model can only work with what is in front of it and what it absorbed during training. When the material it needs is missing from the prompt, the model still produces text, and that text can be wrong in a fluent, confident way. Sufficient context is the idea of making sure the necessary material is present before the model begins answering.
What do you think happens?
You want a model to answer only from a document you supply, so you add one line to the prompt saying to use only the provided material. How much control does that single line give you?
Reveal answer
Less than you might expect. Prompting alone does not reliably enforce the restriction.
Adding a line of instruction is a request for the model to behave a certain way. The source is clear that prompting alone does not reliably guarantee that the model will hold back what it knows. The rest of this article explains why, and what stronger methods exist.
Two Sources an Answer Can Come From
Consider a model that receives a question with no supporting material attached. It has two possible sources for an answer: whatever it absorbed in training, which is its internal knowledge, and whatever you placed in the prompt. When the prompt is empty of the needed facts, the model falls back on internal knowledge.
That internal knowledge may be incomplete, outdated, or simply mistaken about your specific case, and the model has no built-in signal telling it when this has happened. The result is a hallucination: an answer that sounds plausible but is not grounded in reliable information.
The reverse also holds. Including the relevant material in the context tends to improve responses. If you want the model to answer questions about a particular paper, placing that paper in the context gives the model something concrete to draw on, rather than asking it to recall the paper from memory.
Answering a question about a specific paper
A user asks, 'What sample size did the study in this paper use?' The paper is not attached to the prompt.
- 1Check the prompt
The prompt contains only the question. The needed fact is absent.
- 2Model falls back
The model can only use internal knowledge. If it has no reliable memory of this paper, it may still produce a confident number.
- 3Add the paper
The same question is sent again, now with the paper placed in the context.
- 4Model draws on the paper
The model has concrete text to work from, so its answer can be grounded in the document rather than recalled.
Supplying the necessary material changes the model's task from recalling to reading, which is the basic reason sufficient context reduces hallucination.
Context Construction and Its Two Tools
Context construction is the process of gathering the necessary context for a given query. It is the step that happens before the model writes its answer, and its job is to assemble the right material for the question being asked.
There are two broad ways to make sure the context is sufficient. You can provide the necessary context yourself, or you can give the model tools that gather context for it. Context construction is the umbrella term for that second route, and its two named tools are data retrieval and web search.
Read the diagram as a hierarchy of purpose rather than a strict sequence of steps. Context construction is the umbrella. Data retrieval and web search are both tools that serve it, and a RAG pipeline is one concrete way to carry out data retrieval. Each tool is a different means to the same end: assembling sufficient context before the model answers.
| Aspect | Data retrieval | Web search |
|---|---|---|
| Role in context construction | A tool that gathers context for a query | A tool that gathers context for a query |
| Where information comes from | A body of data the system is set up to draw on, such as through a RAG pipeline | The web |
| Concrete example from the source | A RAG pipeline is one way to carry out data retrieval | Named directly as one of the two main tools |
| Common fit | Questions that depend on material you have chosen to make available | Questions that depend on information found publicly online |
The source names both tools as serving context construction. The distinctions in the source sit mainly in where the information comes from.
Restricting a Model to Its Provided Context
Sufficient context helps a model answer well, but some applications need something stricter: the model should answer using only the information it was given. This is common in roleplaying and simulations. Suppose a model is playing a character in a video game. That character should know only about the game's world. It should not be able to answer a question about something outside that world, such as a question about a coffee chain's menu, because the character has no way of knowing it.
Restricting a model this way is harder than providing context. Adding material is additive. Restriction asks the model to hold back knowledge it already has, and the source is clear that this is tricky. The methods it describes fall along a range of reliability, and that range is the key to understanding them.
The first method is prompting. Clear instructions, examples of unanswerable questions, and requests to quote sources can all help. The second is finetuning, which can shape a model toward a particular corpus. Finetuning does not fully solve the problem, because pre-training data may still leak into responses. The strictest method is training exclusively on a permitted corpus. The source describes this as the safest approach, but notes it is often not feasible.
Read the diagram as a progression in how much each method controls what the model knows. Each step further along offers more control, but each also costs more effort or is harder to apply in practice. Prompting sits at the cheap end: it changes what the model is asked to do, not what it has absorbed. Finetuning changes the model's tendencies but, as the source notes, may not stop pre-training material from appearing. Exclusive training changes the knowledge base itself, which is why it is the most reliable and also the hardest to achieve.
Mistakes Beginners Make with Context
Common mistake Assuming a single instruction line guarantees the model will use only the supplied material
What it looks likeAdding 'use only the provided document' to a prompt and treating every later answer as document-grounded.
Why it failsPrompting is a request for behavior. The source states that prompting alone does not reliably enforce the restriction.
Do this insteadSupply the material, ask the model to quote sources, and check answers against the source. Treat the instruction as one layer of control.
Common mistake Assuming a question with no supporting material will simply get no answer
What it looks likeAsking about a specific paper without attaching it, and accepting a fluent reply as fact.
Why it failsThe model still produces text. With the needed material absent, it falls back on internal knowledge that may be incomplete or wrong, and it has no built-in signal that this has happened.
Do this insteadCheck whether the needed information is actually present in the input before trusting the answer.
Common mistake Assuming finetuning removes all of the model's pre-training knowledge
What it looks likeFinetuning a model on a company's documents and expecting it to answer only from those documents.
Why it failsFinetuning can shape a model toward a corpus, yet pre-training data may still leak into responses.
Do this insteadDo not rely on finetuning alone for strict restriction. Test responses to questions outside the permitted material.
Common mistake Confusing context construction with restriction: gathering good context makes answers better, but it does not by itself stop the model from using outside knowledge
Applying the Ideas to a Support Chatbot
A team is building a chatbot that must answer customers only about the company's return policy document. Describe two steps you would take to make sure the answers are supported by that document, and explain why prompting alone would not be enough. Name which of the two context tools would fit the problem if the policy document is stored in the company's own data, and say what the restriction methods can and cannot guarantee.
Hints
- Start with the source of the answer: is the policy text present in the model's input for each question?
- Context construction names two tools, data retrieval and web search. Think about where the policy document lives.
- Consider the restriction ladder: prompting, finetuning, and exclusive training. Which of these is described as reliable but often not feasible?
Key Takeaways
- A model answers from what is in its input and from what it absorbed in training. When the needed material is missing, it can produce a fluent but ungrounded answer, which is a hallucination.
- Context construction is the process of gathering the necessary context for a query. Its two named tools are data retrieval, such as a RAG pipeline, and web search.
- Restricting a model to its provided context is harder than supplying context. Prompting alone does not reliably enforce it, finetuning may still leak pre-training data, and training exclusively on a permitted corpus is the safest but is often not feasible.
- Read restriction methods as a progression of control: each step offers more control over what the model knows, at a higher cost in effort.