Advanced RAG 03 - Hybrid Search BM25 & Ensembles

Hybrid search in retrieval-augmented generation (RAG) combines two retrieval styles: keyword matching and semantic matching. The core idea is to pair a sparse retriever—built on BM25—with a dense retriever—built on embeddings—so results benefit from both exact term overlap and meaning-based similarity. This matters because many real queries contain both “literal” signals (names, exact phrases, specific terms) and “semantic” signals (concepts that may not share the same wording). Hybrid search aims to reduce the failure modes of using only one approach.

BM25 is the keyword component. It’s not a new algorithm; it has existed since the 1970s and 1980s and remains competitive with more modern deep-learning approaches in many search settings. BM25 works by creating sparse representations from word and n-gram counts, using TF-IDF-style weighting (term frequency and inverse document frequency). A practical advantage is speed: sparse keyword scoring is often much faster to compute than dense embedding similarity.

On the semantic side, the dense retriever uses embeddings. In the transcript’s example, OpenAI embeddings are used to embed documents and then store them in a FAISS vector store for similarity search. When a query like “green fruit” is issued, the dense retriever retrieves documents that match the concept even when the exact keyword doesn’t appear—such as returning fruit-related text and “apples and oranges” even if the query words don’t directly occur.

The transcript demonstrates the difference with a small set of documents containing phrases like “I like apples,” “I like computers by Apple,” and “I love fruit juice.” With BM25, a query such as “apple” tends to favor direct keyword matches, which can pull in the “Apple” computer reference depending on how terms appear in the documents. With the dense retriever, a query like “green fruit” shifts results toward fruit-related meaning rather than literal token overlap.

The key step is combining both retrievers using an ensemble retriever. The ensemble takes the sparse BM25 retriever and the dense embedding retriever, then applies a weighting scheme to re-rank and merge their outputs. In the example, hybrid retrieval for “green fruit” elevates fruit-related documents first, while a query like “Apple phones” pushes “I like computers by Apple” to the top because the semantic signal aligns with the intended meaning.

The takeaway is not that hybrid search always beats pure semantic search, but that it often helps when queries include exact words that appear in text—such as people’s names, product terms, or other specific entities—while still benefiting from semantic matching when wording varies. The example is intentionally simplified, but it’s positioned as a template: build a BM25 sparse retriever, build an embedding-based dense retriever (e.g., with FAISS), then combine them with an ensemble and tune weights to match the use case.