Advanced RAG 06 - RAG Fusion

RAG Fusion aims to narrow the gap between what users type and what they actually mean by turning one user query into several targeted search queries, then merging the best retrieval results before the language model writes an answer. Instead of relying on a single vector search lookup, it generates multiple query “views,” runs each through retrieval, and then uses Reciprocal Rank Fusion to rerank and combine the outputs. The payoff is broader, more reliable context—especially when questions are vague or cover multiple angles.

The core workflow starts with query duplication “with a twist.” A single input question is rewritten into multiple related queries (the example uses five), and each rewritten query triggers its own vector search against the knowledge base. Because each query variant tends to surface different documents, the system collects a richer candidate set than any one lookup would provide. Those candidates are then reranked using Reciprocal Rank Fusion, an algorithm designed to combine ranked lists from multiple retrieval runs into a single ordering that favors items that appear consistently well across the different queries.

After reranking, the selected passages are treated as the context for generation. The large language model receives the original user question plus the fused, filtered retrieval results, and then produces a final response that reflects multiple perspectives rather than whatever happened to match the first query formulation.

A key practical insight is that the query rewriting step can be steered to pull different angles of the same topic. The transcript’s example frames this as rewriting one question into variants that emphasize, for instance, an economic perspective versus a public-health perspective—useful when users ask broad questions but want a diverse set of evidence in the answer.

To demonstrate the approach in code, the walkthrough reproduces RAG Fusion using LangChain and Google’s Palm 2 model (with a note that an API key can be obtained via Google’s Make a suite site and that Palm 2 is available for free). The setup uses a Chroma vector database containing precomputed embeddings over a small dataset of scraped Singapore tourist-attraction articles. For embeddings, it uses BGE embeddings, and the notebook either loads an existing Chroma DB or outlines how ingestion would split documents into chunks and compute embeddings.

Retrieval begins with a basic retriever and a simple RAG chain that can already answer questions like “tell me about Universal Studios Singapore,” even when the spelling is off. The RAG Fusion version then adds a query-generation chain: a prompt instructs the model to output multiple search queries from the single user input (the example generates four). Those queries are split out, mapped to the retriever, and their results are merged via Reciprocal Rank Fusion. Debugging traces (via LangSmith) can show the intermediate steps, including the specific rewritten queries.

In the Universal Studios example, the generated queries cover rides, pricing, best things to do, and the best time of year to visit—some highly relevant, others less so. The final generation step uses the fused, filtered context to produce the answer, effectively combining the strongest material retrieved across multiple query formulations. The overall message: RAG Fusion is a retrieval-quality upgrade that mainly works by rewriting and fusing ranked results, not by changing the language model itself.