Long Context Windows in Language Models
For Claude models, the guidance recommends including the entire knowledge base in the prompt, with no need for RAG or similar methods, when the knowledge base is smaller than 200,000 tokens.
A Practical Choice With Your Documents
Suppose you have a body of reference material, such as product manuals, policy documents, or research notes, and you want a language model to answer questions about it. You face a practical design choice before you write any code. You could place the whole body of material inside the prompt you send to the model, an approach often called long context. Or you could use retrieval-augmented generation, usually abbreviated RAG, where the system first searches the material for passages relevant to each question and then sends only those passages to the model.
What do you think happens?
Your knowledge base for a Claude model is about 150,000 tokens, which is smaller than 200,000. Which approach does the guidance recommend?
Reveal answer
Include the entire knowledge base in the prompt, with no need for RAG or similar methods.
For Claude models, the guidance says that when the knowledge base is smaller than 200,000 tokens, the whole thing can go into the prompt. The rest of this article explains why size is the deciding factor and where the rule stops applying.
Full Prompt Versus Retrieval
The two approaches differ in what reaches the model. In the full-prompt approach, the entire body of material is placed inside the prompt, and the model receives all of it along with the question. In the RAG approach, the material is not sent in full. For each question, the system first searches the material for passages that look relevant, and then only those passages are sent to the model along with the question.
Read the diagram as a comparison of steps. The full-prompt path has no search step: the material goes to the model directly. The RAG path adds a retrieval step before the model is called, and that step is what makes RAG a different design. Neither path is automatically better. The guidance examined here tells you which path it recommends for Claude models, and the deciding factor is the size of the knowledge base.
The 200,000-Token Threshold
For Claude models, the guidance recommends including the entire knowledge base in the prompt, with no need for RAG or similar methods, when the knowledge base is smaller than 200,000 tokens.
Anthropic, the developer of the Claude models, gave this guidance in 2024. Its advice is conditional on size. The condition is an upper bound: the knowledge base must be smaller than 200,000 tokens. Here N stands for the size of the knowledge base measured in tokens.
The arrow reads as the guidance's recommendation when the condition on the left holds. This expression is a plain restatement of the source's rule for Claude models. It is not a general law of retrieval systems, and it says nothing on its own about what to do when the condition fails.
The diagram has one clear path and one open branch. If the size is below the threshold, the recommended path is to include the whole knowledge base in the prompt. If it is not below the threshold, this passage gives no direct instruction, so the comparison has to be made with more information. Notice that the open branch includes the exact threshold value, because a knowledge base of exactly 200,000 tokens is not smaller than the threshold.
Tracing Three Knowledge Bases Through the Rule
Applying the threshold to three collections
A Claude-based assistant will answer questions from three collections of reference material, sized at 150,000 tokens, 900,000 tokens, and exactly 200,000 tokens. For each one, state whether it falls under the threshold and which approach the guidance recommends.
- 1Check the first collection
The size 150,000 is smaller than 200,000, so the condition N < 200,000 holds.
- 2Read the recommendation for that case
When the condition holds, the guidance recommends including the entire knowledge base in the prompt, with no need for RAG or similar methods.
- 3Check the second collection
The size 900,000 is well above 200,000, so the condition does not hold. The passage does not tell you which approach to choose here.
- 4Check the third collection
The size is exactly 200,000. It is not smaller than the threshold, so it falls outside this recommendation even though it sits right at the boundary.
The 150,000-token collection gets the full-prompt recommendation. The 900,000-token collection and the 200,000-token collection both fall outside this guidance, so the team must evaluate the options using other criteria.
The example is not a claim that RAG wins for large collections. The passage supports only the small-collection conclusion. For the 900,000-token collection, the right next step is to weigh the approaches against whatever other criteria matter to your project, because this guidance does not settle the question.
Where the Guidance Applies
The rule has a specific scope, and it helps to separate what is inside that scope from what is outside it. The concept map below shows the parts of the rule and the situations that fall outside it.
Inside the scope are Claude models, a knowledge base smaller than 200,000 tokens, and a recommendation about placing the entire knowledge base in the prompt. Outside the scope are other models, which the guidance does not address, and collections of 200,000 tokens or more, which the guidance does not decide. The source also notes that it would be helpful if other model developers gave similar guidance on RAG versus long context for their own models. Until such guidance exists for a given model, you cannot borrow the Claude threshold and treat the decision as settled.
Tokens and Their Approximate Page Equivalent
The source gives a rough human scale for the boundary. The 200,000-token figure is described as about 500 pages of material. This is an approximation, and the page count is an estimate that depends on how dense the pages are. The token count is the number the guidance actually uses to draw the line, so the page figure is only a way to picture the size.
| Aspect | Token count reading | Approximate page reading |
|---|---|---|
| Role in the guidance | The figure the guidance uses to set the condition | A rough human scale for the same boundary |
| Value for the boundary | 200,000 tokens | About 500 pages |
| Precision | The threshold the rule is written against | An estimate that depends on page density |
| Use when deciding | Compare the knowledge base size against 200,000 tokens | Only to picture roughly how much material that is |
The token count is the decision figure; the page count is an approximate equivalent.
Common Misreadings of the Threshold
Common mistake Treating 500 pages as the rule itself
What it looks likeDeciding that a 480-page manual falls under the rule because it is close to 500 pages, without measuring its token count.
Why it failsThe source says the token count is the figure the guidance uses. The page figure is an approximate equivalent that depends on how dense the pages are.
Do this insteadMeasure the knowledge base in tokens and compare that number to 200,000.
Common mistake Reading the threshold as inclusive
What it looks likeConcluding that a knowledge base of exactly 200,000 tokens gets the full-prompt recommendation.
Why it failsThe condition is that the knowledge base is smaller than 200,000 tokens. A collection at exactly the threshold is not smaller than it.
Do this insteadTreat 200,000 tokens and above as outside this recommendation.
Common mistake Applying the Claude threshold to every model
What it looks likeUsing the 200,000-token rule to choose an approach for a model from a different developer.
Why it failsThe guidance is specific to Claude models and gives no recommendation for other models.
Do this insteadLook for guidance from that model's developer or evaluate the approaches yourself.
Common mistake Assuming RAG is the right choice for large collections
What it looks likeDeciding that a 900,000-token collection must use RAG because the knowledge base is large.
Why it failsThe passage does not say which approach to choose above the threshold. It only supports the small-collection conclusion.
Do this insteadEvaluate both approaches with other criteria before deciding.
Applying the Rule to Your Own Material
A Claude-based tool will answer questions from a collection of policy documents measured at 180,000 tokens. Using the threshold, state whether the guidance recommends including the whole collection in the prompt, and explain what you would do if the collection grew to 240,000 tokens.
Hints
- Compare the token count directly to 200,000. Is 180,000 smaller than that?
- A collection of 240,000 tokens is outside the condition, so the passage gives no direct instruction for it.
- Remember that the page equivalent of 500 pages is an approximation, not the decision figure.
Key Takeaways
- For Claude models, the guidance recommends including the entire knowledge base in the prompt, with no need for RAG or similar methods, when the knowledge base is smaller than 200,000 tokens.
- The threshold is an upper bound, so a knowledge base of exactly 200,000 tokens or more falls outside this recommendation, and the passage gives no direct instruction for it.
- The token count is the figure the guidance uses. The figure of about 500 pages is an approximate human-scale equivalent that depends on page density.
- The guidance applies to Claude models only, and the source notes that other model developers would benefit from giving similar guidance for their own models.