Vectorless RAG Tutorial With PageIndex-No VectorDB And Chunking Required

Vectorless RAG replaces the usual “chunk → embed → store in a vector database → similarity search” pipeline with a document-structure index that an LLM can navigate directly. Instead of building embeddings and running cosine similarity over a vector store, PageIndex builds an LLM tree from a PDF, stores section-level summaries in a JSON tree index, and then uses an LLM to reason over that hierarchy to retrieve the most relevant sections—complete with page citations.

In traditional vector RAG, long PDFs are split into chunks, each chunk is embedded using an embedding model (OpenAI, Gemini, etc.), and the resulting vectors are stored in a vector database. At query time, the user question is embedded, the system performs similarity search to fetch the top matching chunks, and the retrieved text becomes context for the LLM to generate an answer.

Vectorless RAG removes the vector database entirely. The core mechanism is an “LLM tree builder” that turns a document into a hierarchy of sections (for example, introduction → AI → machine learning → deep learning), where each node contains an LLM-generated summary of the content for that section. The output is a JSON tree index representing the document’s structure and node summaries. When a user asks a question, the LLM receives this JSON tree index as context, traverses the hierarchy, selects the nodes most likely to contain the answer, and then extracts the corresponding section content to produce a response. The retrieval loop is explicit: the system reasons about which nodes to use, checks whether the extracted content is sufficient, and if not, it iterates by selecting additional nodes.

A key detail is how the system handles documents with or without a table of contents (TOC). If a TOC exists, it can use it as an index of sections and page numbers. If no TOC is available, the system performs TOC detection by scanning for headers on end pages; when that fails, the LLM reads pages to infer headings and structure. Retrieval then becomes “section-aware splitting” rather than token-count chunking. The result is that the LLM can retrieve logically bounded sections (e.g., the deep learning section) instead of arbitrary text windows that may mix topics.

The tutorial demonstrates this using the PageIndex library and a hosted chat interface (chat.pageindex.ai). A sample PDF is uploaded, the tree index is built asynchronously (roughly 30–90 seconds for a ~50-page document), and the resulting tree is inspected as JSON with node IDs, titles, page ranges, and node summaries. In the example query—“What is the syllabus covered in modern LLM fine-tuning?”—the LLM tree search identifies the relevant node IDs (such as nodes corresponding to the fine-tuning sections), extracts the associated content, and then generates an answer with citations formatted as section title and page number.

Finally, the walkthrough provides end-to-end Python-style steps: load environment variables (PageIndex API key and OpenAI API key), upload the PDF, poll for tree readiness, fetch the tree structure, run an LLM tree search that selects nodes via a prompt, and generate the final answer using only the retrieved context. The practical takeaway is a lower setup burden—no vector database, no embedding pipeline—at the cost of relying on structured indexing and LLM reasoning over the document hierarchy.