Advanced RAG 02 - Parent Document Retriever

Parent document retrievers fix a common RAG tradeoff: embeddings need to be specific enough to find the right passage, but the language model needs broader surrounding context to answer coherently. Instead of embedding only the same chunks that get returned, the approach embeds smaller “child” chunks for retrieval while returning the larger “parent” chunks that contain the full context. The result is a tighter semantic match from embeddings, paired with richer text for in-context learning.

In conventional RAG, documents get split into chunks, each chunk is embedded, and retrieval returns those same chunks. That can work, but chunk size strongly affects embedding quality. If a chunk is too large, its embedding can become “washy” and less specific—especially in long documents that mix multiple topics. If a chunk is smaller, embeddings become more targeted (for example, a chunk focused on “ads revenue” vs. one focused on “Bard revenue”). The problem arises when the model needs to compare or synthesize across multiple details: embeddings must be specific for retrieval, yet the model benefits from seeing a larger slice of the original document.

Parent document retrievers address this by splitting twice. First, the original document is divided into larger parent chunks. Then each parent chunk is further split into smaller child documents, and embeddings are computed for the child documents. At query time, the system embeds the question, retrieves the most relevant child chunks, and then maps those matches back to their parent chunks. The language model ultimately receives the parent text, not the small child fragments—so it gets more context without sacrificing retrieval precision.

LangChain implementation demonstrates two practical modes. The first mode returns the full original document as the parent. This works well when each source document is already short enough to be safely included at the end. In the notebook example, two scraped LangChain blog posts are stored as full documents, then split into smaller recursive character chunks for embedding. A similarity search for “What is LangSmith?” returns small matching chunks, which are then expanded to return the entire blog post. The returned context jumps from a few hundred characters to roughly 11,600 characters for the full post.

The second mode uses multi-layer chunking for long documents. Instead of returning the entire source, it returns larger parent chunks that are still manageable for the language model. The example sets parent chunks to about 2000 characters and child chunks to about 400 characters. After indexing, the system retrieves child matches but returns only the relevant parent chunks—yielding multiple overlapping big chunks (e.g., four child matches collapsing into two parent chunks) that provide enough coverage for a coherent answer.

Finally, a retrieval QA chain ties it together: a question is answered by feeding the retrieved parent chunks into a language model (OpenAI in the example). The output for “What is LangSmith?” becomes a complete definition, supported by the broader parent context rather than isolated fragments. The core takeaway is that parent document retrievers let systems keep embeddings granular while still giving the language model the surrounding text it needs to produce accurate, well-formed responses.