Embedding-Based Retrieval

Suppose you type a few words into a search box and get back a list of results. Somewhere behind that list is a decision the system had to make for every item it could show you: how well does this item match what I asked for? Retrieval is the name for the family of techniques that make that decision and act on it.

This article introduces the idea, shows where it sits in the larger picture, and sets up the two mechanisms that the rest of this chapter builds on. Keep the search box in mind as you read, because almost every idea here can be traced back to what happens when a query goes in and an ordered list comes out.

The important word in that definition is ranking. Retrieval does not simply hand back a yes or no for each document. It orders the documents so that the most relevant ones come first. Every retrieval method has to answer the same underlying question, which is how relevant each document is to the query, and then sort the documents on that answer.

Follow one query through the operation to see where each piece fits. The query and the document collection both feed into the step that computes a relevance score for each document. The scores then determine the final order of the list you see.

Notice which part of this pipeline changes from one retrieval method to another. The input and the final sorting are shared. The middle step, where the relevance score is computed, is the only part that differs. This is the first clue to the chapter's central lesson: the goal stays fixed while the scoring method varies.

It is easy to associate retrieval with retrieval-augmented generation (RAG), because RAG pipelines depend on fetching relevant material before a language model writes its answer. But retrieval is not unique to RAG. Information retrieval is an old idea, and it already forms the backbone of search engines, recommender systems, and log analytics. These systems existed long before RAG did.

The practical consequence is worth remembering. Many retrieval algorithms were developed for traditional retrieval systems, and those same algorithms can also be reused inside RAG. When you meet a retrieval method in a RAG setting, it may have a long history in search and recommendation work behind it.

The chapter introduces two common retrieval mechanisms as a starting point: term-based retrieval and embedding-based retrieval. Each one is a different answer to the scoring question raised earlier. Term-based retrieval answers it using the terms that appear in the query and the documents. Embedding-based retrieval answers it using a learned representation of the query and of each document, so that relevance is judged by how closely those representations match. This article stays at the level the chapter introduces; the mechanics are developed in later sections.

The comparison points to the central lesson of this section. Term-based and embedding-based retrieval are two answers to one question. Both produce a ranked list, and the difference lies entirely in how each computes the relevance score that the ranking depends on. The next part of the chapter takes up a broader way of grouping such methods, known as sparse versus dense retrieval.

Before reading the answer, decide what a retrieval system actually hands back for each document it considers.