Embedding-Based Retrieval and Vector Search

Imagine you search a large document collection for the word transformer. A keyword search will return every page that contains that word. Some of those pages may describe an electrical device, some may describe a neural network design, and some may be about a film. The word matched, yet the meaning you wanted may be missing from the results. A search system that depends on only one retrieval method inherits that method's weakness, and no single method is free of blind spots.

The practical response is to combine retrieval methods so that each one covers for the other's blind spots. The source states the motivation directly: each retrieval algorithm has its own advantages, so a production retrieval system typically combines several approaches rather than committing to one. The rest of this article explains the two main ways of combining them.

The source identifies two arrangements. In a sequential arrangement, different algorithms are used in a chain, where one feeds the next. In a parallel arrangement, often called an ensemble, several algorithms run on the same query and their outputs are merged. The next two sections follow each arrangement in turn, starting with the chain.

In a sequential arrangement, the first retriever is chosen for being cheap and reasonably good at finding a broad set of candidates, even if it is not very precise. The source gives a term-based system as the example of this first step. The second retriever is chosen for being more precise, even though it costs more to run. The key point is that the second retriever only needs to look at the candidates the first step produced. That restriction is what makes its expense affordable.

The second step in this chain has a specific name: reranking. Reranking reorders the candidates that already exist so that the ones actually related to the query rise to the top. Each stage hands a smaller and more relevant set forward, so the expensive step never has to examine the entire collection.

The other arrangement does not chain the algorithms. Instead, several retrievers receive the same query at the same time, and each produces its own ranking of documents. A retriever works by ranking documents according to their relevance scores for the query, so each retriever's output is a ranked list. The retrievers never see each other's output while they work. They are independent, which is why they can run together.

The diagram makes one point clear: the merging happens after the retrievers finish. Because each retriever returns its own ranked list, the remaining task is to combine those lists into one final ranking. The source names one combination method for this job, reciprocal rank fusion.

Reciprocal rank fusion, abbreviated RRF, is the algorithm the source gives for combining different rankings. It does not look at the raw relevance scores each retriever produced. It looks only at where each document landed in each list. Each document receives a score based on its rank from a given retriever, and the higher the document ranks, the higher that score is.

Under this rule, a document ranked first receives $\frac{1}{1} = 1$, and a document ranked second receives $\frac{1}{2} = 0.5$. A document ranked third receives $\frac{1}{3}$, which is about 0.333. The score falls as the rank number grows. Because the rule depends only on position, a retriever whose raw scores are on a different scale from another retriever's can still be combined with it, without first converting both scales to a common unit.

Notice why D1 beats a document that ranks first in only one list. A document ranked first in one list and absent from the other gets only its single score. A document that appears near the top of both lists collects two contributions, so the total grows. This is why agreement between retrievers is rewarded.

The two retrievers in the ensemble above are chosen for different strengths. A term-based method matches the words in the query against the words in documents. An embedding-based method compares vectors, so that a query and a document can be matched through their representation rather than only through shared words. The source's keyword example shows where word matching falls short: the word matched, yet the intended meaning may be missing.