Advanced & Modular RAG
retrievalUpdate 07/2026: The 2026 consensus in one sentence: naive RAG is dead, retrieval is not. Two things changed since our last entry:
- Long-context hybrid architectures are standard: with context windows in the million-token range, small corpora can skip retrieval entirely - but for large or dynamic knowledge bases, retrieval remains essential for cost, latency and grounding. Production systems combine both: retrieval narrows down, long context carries more of it.
- Agentic RAG is increasingly used in production and works well: embedded in agent loops, the retrieve → rethink → retrieve cycle fits naturally - the agent reformulates queries, evaluates results and retrieves again until the context suffices. RAG evolves from a fixed pipeline into a "context engine" that agents query iteratively. See also Context Engineering and Smart Vector Database Usage for the retrieval-side defaults (hybrid search + reranking).
The RAG-paradigm framing of our previous entry (Gao et al., 2023) is still useful vocabulary, but read it as history of the field rather than current architecture guidance.
RAG ("Retrieval-Augmented Generation") is a now well established technique to use the cababilities of Large Language Model (LLM) together with information from external data stores (like custom knowlege). It enables LLMs to access external, up-to-date and specific informations. It also reduces "hallucinations" and enables domain-specific knowledge integration.
The implementation of advanced RAG and modular RAG systems have prooven to produce good results. When we talk about "advanced RAG" and "modular RAG", we are referring to the different RAG Paradigms, that are introduced in Source (Gao et al.).
The following graphic also illustrates the different RAG paradigms:
Advanced RAG
In an advanced RAG paradigm, pre-retrieval and post-retrieval phases are added to the naive RAG paradigm.
The phases of an advanced RAG system:
- Pre-retrieval — Query rewriting, query entity extraction, query expansion, etc.
- Retrieval of relevant context
- Post-retrieval: Reranking, pruning, extending, etc.
- Answer generation
GraphRAG
Utilizes knowledge graphs during indexing and retrieval. See the extra article on GraphRAG for a deep dive.
Modular RAG
A modular RAG system contains more complex patterns which involve orchestration and routing of the user query. Modular RAG and AI Agent pattern overlap.
Agentic RAG
Agentic RAG introduces a reasoning agent that actively evaluates, reconciles, and refines retrieved information to provide more accurate data for the final response.
So instead of just retrieving and augmenting, an agent like component has access to retrieval tools (e.g. function calls) - and takes the task to retrieve and refine the knowlege - before it is passed to the main LLM call.
This can be powerful - but the agentic layer is a significant increase in complexity and cost, and managing the retrieval, tool access and decision-making requires substantial engineering effort. This extra steps also lead to increased latency and the additional component is another source for potential misinterpretation and halluzination in the overall system.