What is modular RAG?

Modular RAG is a RAG architecture in which retrieval and generation are organized as independent, reusable modules that can be combined, routed, or repeated depending on the task.

Instead of forcing every query through one fixed linear pipeline, modular RAG allows the system to choose different processing paths.

What can be modular?

A modular RAG system may contain components such as:

• Query rewriting
• Routing
• Retrieval
• Metadata filtering
• Reranking
• Context compression
• Memory
• Generation
• Validation or feedback

These components can be assembled differently for different queries.

Simple example

A basic RAG pipeline might always do this:

Query → Retrieve → Generate

A modular system could instead route queries based on what they need:

                 ┌→ Vector retrieval ─→ Rerank ─┐
User query → Router                            ├→ Generate
                 └→ Database lookup ───────────┘

Another workflow might rewrite a difficult query, retrieve from several sources, combine the results, and repeat retrieval if the evidence is insufficient.

How is modular RAG different from advanced RAG?

Advanced RAG improves the quality of a conventional pipeline with techniques such as query rewriting, hybrid retrieval, and reranking.

Modular RAG goes further by treating those capabilities as configurable building blocks. The workflow itself can be linear, conditional, branching, or iterative rather than fixed.

This makes modular RAG useful for applications where different questions require different retrieval strategies or where the system needs more control over how information flows between components.

However, the added flexibility also increases orchestration, evaluation, and operational complexity.

For the previous stage in this progression, see “What is advanced RAG?”.

In short, modular RAG builds RAG from reusable components that can be dynamically arranged into different retrieval and generation workflows instead of using one fixed retrieve-then-generate pipeline.