K-Nearest Neighbors for Retrieval

Imagine you search a large document collection for the word transformer. A keyword search will happily return every page containing that word, but those pages may be about an electrical device, a neural network design, or a film. The word matched, yet the meaning you wanted may be missing. A search system that relies on only one retrieval method inherits that method's weakness.

The practical response is to combine retrieval methods so that each one covers for the other's blind spots. The source states the motivation directly: each retrieval algorithm has its own advantages, so a production retrieval system typically combines several approaches rather than committing to one. The rest of this article follows the two main ways of combining them.

The phrase hybrid search describes the combination itself, not a single algorithm. Once that is clear, the real design question becomes how the methods are arranged. Term-based retrieval finds documents by the words they contain. Embedding-based retrieval, which the source also calls vector search, pulls out content that matches the meaning of the query. Each approach can be placed in a chain or run side by side.

In a sequential arrangement, the first retriever is chosen for being cheap and reasonably good at finding a broad set of candidates, even if it is not very precise. The source gives a term-based system as the example of this first step. The second retriever is chosen for being more precise, even though it costs more to run. It only needs to look at the candidates the first step produced, and that is what makes its expense affordable.

Reading the pipeline from left to right shows why the arrangement is efficient. Each stage hands a smaller and more relevant set forward, so the expensive step never has to examine the entire collection. The second step has a specific name: reranking. Reranking does not search from scratch. It reorders the candidates that already exist so that the ones actually related to the query rise to the top.

The source offers a second chain with a different shape. Suppose the query is: Who is responsible for the most sales to X? The first step uses the keyword X to fetch all documents associated with that customer. The second step then uses vector search to pull out the context matching the question about who is responsible for the most sales.

The other arrangement does not chain the algorithms. Instead, several retrievers receive the same query at the same time, and each produces its own ranking of documents. The source describes this as using the algorithms in parallel as an ensemble. A retriever works by ranking documents according to their relevance scores for the query, so each retriever's output is a ranked list. The remaining task is to combine those lists into one final ranking.

The diagram makes one point clear: the two retrievers never see each other's output while they work. They are independent, which is why they can run together. The merging happens afterward, and that merging is the job of a combination method. The source names one such method, reciprocal rank fusion.

Reciprocal rank fusion, abbreviated RRF, is the algorithm the source gives for combining different rankings. It does not look at the raw relevance scores each retriever produced. It looks only at where each document landed in each list. Each document receives a score based on its rank in a given retriever's list, and the higher the document ranks, the higher that score is.

Because the rule depends only on position, a retriever whose raw scores are on a different scale from another retriever's can still be combined with it. There is no need to convert the scales to a common unit first. A document's score from one list is simply determined by where it sits in that list.

The two retrieval approaches decide that a document matches a query in different ways, and that difference is the reason hybrid search exists. The table below contrasts them using only the behavior the source describes.