Information Retrieval Fundamentals
A RAG system has two components: a retriever that fetches relevant information from external memory sources, and a generator that writes a response based on the retrieved information.
A Question the Model Cannot Answer Alone
Imagine a user asks a question that a language model cannot answer reliably from memory alone, because the answer lives in a document the model never saw during its own training. A retrieval-augmented generation system, usually called a RAG system, handles this situation by splitting the job into two cooperating parts.
Before looking at either part on its own, it helps to follow one question from start to finish. As you follow it, notice which part touches the outside material and which part writes the reply. That single question-and-answer journey is the easiest way to hold the whole system in mind.
Following a Query Through the System
The easiest way to understand a RAG system is to follow the direction of data. The query enters the retriever. The retriever searches external memory and hands back relevant material. That material, together with the query, is what the generator consumes. The generator then emits the response.
Read the diagram carefully. The external memory is reached only through the retriever, and the response is produced only after the generator has received the retrieved material. The generator also receives the original query, so it knows what it is answering.
What do you think happens?
Suppose the retriever returns passages that have nothing to do with the query. What happens to the final response?
Reveal answer
The generator has nothing useful to work with, so the response cannot draw on relevant external facts.
The retriever is the only link between the system and outside memory. The quality of what it fetches limits what the rest of the system can say, so off-topic passages leave the generator without useful material.
Two Components and Their Responsibilities
A RAG system has two components: a retriever that fetches relevant information from external memory sources, and a generator that writes a response based on the retrieved information.
Tracing a Question Through an HR Assistant
An HR assistant answering a policy question
An employee asks: How many days of parental leave does the company offer? The model was never trained on the company's internal policy handbook, so it cannot answer reliably from memory alone. Trace what happens in the RAG system.
- 1The query enters the retriever
The question arrives at the retriever, which is the component responsible for locating outside information relevant to the query.
- 2The retriever searches external memory
In this setup, the external memory is the company's policy documents. The retriever looks for passages that look relevant to parental leave and sets aside the rest.
- 3Retrieved material is handed on
The relevant policy passages are passed along. The retriever does not write an answer for the employee at this stage.
- 4The generator receives query and material together
The generator gets the original question and the retrieved policy passages. Its answer should draw on the passages, not on whatever the model happens to know about leave policies in general.
- 5The generator writes the response
The generator composes the reply the employee reads, and the facts it relies on came from the handbook retrieved at the moment the question was asked.
The employee receives a response grounded in the retrieved policy text. If the retriever had returned unrelated passages, the generator would have had nothing useful to draw on.
Training the Two Parts Apart or Together
A RAG system has two trainable parts, and there are two broad ways to prepare them. They can be trained separately, with each component improved on its own, or they can be trained together so that the whole system is tuned as a unit. The source describes both patterns and explains how practice has shifted over time.
Mistakes Beginners Make with RAG Components
Common mistake Thinking the generator is the part that reaches into external memory.
Why it failsThe retriever is the component that sits between the question and outside material and fetches relevant information.
Do this insteadAssign searching of external memory to the retriever, and assign writing of the response to the generator.
Common mistake Assuming the generator's facts come only from what the model learned during training.
Why it failsThe generator's response is meant to draw on the information the retriever fetched, not on whatever the model happens to know.
Do this insteadRemember that the facts the generator leans on were supplied from external memory when the question was asked.
Common mistake Ignoring the quality of the retriever's output.
What it looks likeA retriever returns passages on an unrelated topic, and the generator is asked a specific policy question.
Why it failsThe retriever is the only link to outside memory, so its output limits what the rest of the system can say.
Do this insteadJudge a RAG system's answers partly by whether the retriever is fetching relevant material in the first place.
Common mistake Believing end-to-end fine-tuning is the standard way to build a RAG system.
Why it failsThe source says end-to-end fine-tuning is less common in modern practice, while separate training with off-the-shelf parts is common.
Do this insteadTreat joint training as a powerful option that is used less often, not as the default starting point.
Practice Tracing the Pipeline
A user asks a RAG system a question about a product manual that the language model never saw in training. Describe, in order, where the query goes, what the retriever does, what the generator receives, and what the user finally reads. Then say which component is responsible for the facts in the final answer.
Hints
- Start with the direction of data: the query enters the retriever first.
- Ask which component is the only one that reaches external memory.
- The generator receives two inputs: the original query and the retrieved information.
Key Takeaways
- A RAG system has two components: a retriever that fetches relevant information from external memory sources, and a generator that writes a response based on the retrieved information.
- Data flows from the user query to the retriever, then from the retrieved material to the generator, which produces the final response. External memory is reached only through the retriever.
- The quality of the retriever's output limits what the generator can say, because the generator works from the material it is given.
- The retriever and generator can be trained separately, which is common in modern RAG systems that use off-the-shelf parts. End-to-end fine-tuning trains the whole system together and can improve performance significantly, though it is less common in modern practice.