Chunking Documents for Retrieval
The generated context describes where the chunk sits in the document. It is not a summary of what the chunk says. The chunk already contains its own content, so the added text contributes location information that the chunk lacks.
Why Cutting a Document Causes Trouble
Imagine a long document that has been cut into many small pieces. Each piece, called a chunk, is easier to search than the whole document. A search system can compare a question against a small piece of text and decide whether that piece is relevant. The cost of cutting, however, is that a chunk loses its surroundings.
A chunk often refers to things that were said elsewhere. It may mention a product, a person, or a time period that was named several paragraphs earlier in the document. When the chunk is read alone, those references may be unclear. The words are still there, but the reader, or the search system, no longer knows which product, person, or period is meant.
What do you think happens?
If a search system sees only the chunk text, what information could it be missing about where that text came from?
Reveal answer
It could be missing which document the text belongs to and where that text sits inside the document, such as the section or the topic that was introduced earlier.
The chunk's own words may be accurate but incomplete. The missing piece is location: the document and the place within it that the chunk came from.
Reading a Chunk Alone
Consider what a retriever actually receives. If indexing is based only on the chunk, then the index holds the chunk's sentences and nothing more. Any question whose words match the chunk can find it, but any question that depends on context outside the chunk has nothing to match against. A question that names the product line, the store, or the policy may never reach a chunk that never mentions those names.
The remedy described in the source is to generate a short piece of context for each chunk before it is indexed. That context situates the chunk within the overall document. It is then added to the chunk so the retrieval system sees both together. The approach aims to improve the search retrieval context, meaning the information the retriever has available when it judges whether a chunk matches a query.
Generating a Short Context
The source shows the request that is made for each chunk. The full chunk is presented to a model along with an instruction. The instruction asks for a short, succinct context that places the chunk within the overall document. It also asks that the answer contain only that succinct context and nothing else.
The generated context describes where the chunk sits in the document. It is not a summary of what the chunk says. The chunk already contains its own content, so the added text contributes location information that the chunk lacks.
Keeping the output short matters. The context is going to be attached to the chunk, so a long preamble would add noise rather than location information. A brief statement naming the document and the section is the intended size.
Prepending the Context to the Chunk
Once the context has been generated, it is prepended to the chunk. Prepending means placing it at the front. The result that moves forward is the context followed by the original chunk text. This joined result is called the augmented chunk, and it is the unit that the retrieval algorithm indexes.
The original chunk is not discarded. The context is joined to it, and the original text remains intact inside the augmented chunk. Keeping the original chunk unchanged while adding the context is central to the approach.
Read the pipeline from top to bottom. The document is divided first, so each chunk exists as a separate piece. Each chunk then receives its own generated context. That context is prepended, and the augmented chunk is what reaches the index. The step that matters most for retrieval is the final one, because the index stores the augmented text, not the raw chunk.
Tracing One Returns Policy Chunk
The following example is an original illustration of the idea. It is not a case taken from the source material. Suppose a document is the returns policy for an online store, and one chunk reads only: Requests must be submitted within 30 days of delivery.
Augmenting a returns policy chunk
A chunk from an online store's returns policy says: Requests must be submitted within 30 days of delivery. Trace what is indexed when context generation is applied.
- 1Read the chunk alone
The sentence states a time limit, but it never says which product line, store, or policy it belongs to.
- 2Generate a short context
The context names the document and the section, for example: this chunk comes from the returns section of an online store's policy and states the time limit for submitting a return request.
- 3Prepend the context
The context is placed at the front, followed by the original sentence, unaltered.
- 4Index the augmented chunk
The retrieval algorithm indexes the joined text, so the index holds both the location information and the original sentence.
The indexed text contains the original 30-day sentence plus a statement of where it sits in the returns section of the policy.
Notice what changed and what did not. The chunk's original sentence is still present and unaltered. What has been added is information about location in the document. A query asking about the return window for an online store now has more matching words available in the indexed text, which is the kind of improvement in search retrieval context the source describes.
Where the Context Sits
It helps to think in containment. The overall document contains many sections, and each section contains chunks. The augmented chunk is a chunk with its context attached at the front. The context itself refers back to the document and section, so it points upward to the larger structure while the original chunk text stays at the core.
The context sits in front of the chunk text, not around the whole document. The document is not copied into the index. Only a brief statement of location is attached to each chunk, and the chunk's own words remain the body of the indexed unit.
Mistakes When Applying Context Generation
Common mistake Writing the context as a summary of the chunk's content
What it looks likeContext: the chunk explains that refunds take five business days.
Why it failsThe source says the context should be about location in the document rather than a summary of the chunk's content. The chunk already states its own content.
Do this insteadWrite a brief statement of where the chunk sits, such as the document and section it belongs to.
Common mistake Replacing the original chunk with the context
What it looks likeIndexing only: this chunk comes from the returns section of an online store's policy.
Why it failsThe original chunk text carries the actual information a query needs to match. Keeping the original chunk intact while adding context is central to the approach.
Do this insteadJoin the context to the original chunk so the augmented chunk contains both.
Common mistake Letting the generated context grow into a long preamble
Why it failsThe context is attached to the chunk, so extra length adds noise rather than location information.
Do this insteadKeep the generated output short and limited to the succinct context the instruction requests.
Common mistake Assuming the index stores the raw chunk after generation
Why it failsThe retrieval algorithm indexes the augmented chunk, which is the context joined to the chunk.
Do this insteadWhen tracing the pipeline, follow the augmented chunk to the index, not the raw chunk.
Trace It Yourself
A document is a user guide for a kitchen appliance. One chunk reads: Clean the filter once a month. Write a short context for this chunk that places it within the document, then write the augmented chunk exactly as it would be indexed. Explain why your context does not summarize the cleaning instruction.
Hints
- Ask which section of the guide this instruction belongs to, and what the guide is about.
- The context should answer where the text is located, not what the sentence says.
- The augmented chunk is the context placed at the front, followed by the original sentence unchanged.
Key Takeaways
- A chunk is easier to search than a whole document, but cutting a document can separate a chunk from the earlier references that give it meaning.
- A short generated context is produced for each chunk to situate it within the overall document, and the instruction asks that the output contain only that succinct context.
- The generated context is prepended to the chunk, and the resulting augmented chunk, with the original text kept intact, is what the retrieval algorithm indexes.
- The context should describe location in the document, not summarize the chunk's content, so that the retriever gains location information it would otherwise lack.