Sparse versus Dense Retrieval

Suppose you type a few words into a search box and get back a list of results. Somewhere behind that list is a decision the system had to make for every item it could show you: how well does this item match what I asked for? Retrieval is the name for the family of techniques that make that decision and act on it. This article introduces the idea, shows where it sits in the larger picture, and sets up the two mechanisms that the rest of this chapter builds on.

Before reading on, pause and ask yourself a simple question: if a system had to order three documents for one query, what would it need to know about each document? Keep your answer in mind. The rest of the article shows how the answer fits into a single repeating operation.

At its core, retrieval works by ranking documents based on their relevance to a given query. The important word is ranking. Retrieval does not simply hand back a yes or no for each document. It orders the documents so that the most relevant ones come first. Every retrieval method has to answer the same underlying question, which is how relevant each document is to the query, and then sort on the answer.

Read the diagram from the top. The query and the document collection both feed into the step that computes a relevance score for each document, and the scores determine the final order. The part that changes from one retrieval method to another is the middle step, where the score is computed. Everything before and after that step is shared across methods, which is why the scoring step is the place to look when you want to understand how two retrieval methods differ.

It is easy to associate retrieval with retrieval-augmented generation (RAG), because RAG pipelines depend on fetching relevant material before a language model writes its answer. But retrieval is not unique to RAG. Information retrieval is an old idea, and it already forms the backbone of search engines, recommender systems, and log analytics. These systems existed long before RAG did.

The practical consequence is useful to remember. Many retrieval algorithms were developed for traditional retrieval systems, and those same algorithms can also be reused inside RAG. When you meet a retrieval method in a RAG setting, it may have a long history in search and recommendation work behind it.

Two words appear constantly in this area, and they are not quite the same. Retrieval is typically limited to one database or system. Search involves retrieval across various systems. The difference is one of scope: retrieval describes finding relevant items within a single source, while search describes the wider act of looking across several sources.

The source introduces two common retrieval mechanisms as a starting point. Each one is a different answer to the scoring question raised earlier: how well does this document match the query? The two are described below, and the comparison that follows lines them up side by side.

The comparison points to the central lesson of this section. The goal stays fixed while the scoring method varies. Term-based and embedding-based retrieval are two answers to one question, and the next section of the chapter takes up a broader way of grouping such methods, known as sparse versus dense retrieval. Notice that both methods start from the same query and the same documents. They diverge only at the scoring step, and the final sorted list is produced the same way for both.