Agentic RAG: Self-RAG, Corrective RAG Patterns

Short answer: in classic RAG, retrieval happens once before generation. In agentic RAG, the model decides whether to retrieve, what to retrieve, judges whether what came back was good enough, and retrieves again if it was not. That loop measurably improves answers on hard questions and makes cost and latency unpredictable, which is the trade you are actually deciding on.

Most systems described as agentic RAG are not. If retrieval still happens once, at a fixed point, with a fixed query, it is a retrieval pipeline with a language model attached — which is fine, and often correct.

What makes retrieval agentic

Four capabilities, and a system needs at least the first two to earn the label:

Query planning. The user’s question is rarely a good search query. Agentic systems rewrite it, decompose it into sub-questions, or decide it needs no retrieval at all. “How does our refund policy compare to last year’s?” is two retrievals and a comparison, not one lookup.

Retrieval evaluation. After retrieving, the system judges whether the passages actually answer the question rather than assuming they do. This is the single highest-value addition, because most bad RAG answers come from confidently composing over irrelevant context.

Multi-hop reasoning. Some questions need the answer to a first retrieval before the second query can even be formed. A fixed pipeline cannot express that.

Self-reflection. Checking the draft answer against the retrieved evidence before returning it, and retrieving again if the answer is not supported.

The named patterns, briefly

Self-RAG trains a model to emit reflection tokens that decide when to retrieve, whether passages are relevant, and whether the output is supported by them. The appeal is that it is one model doing all of it, with no separate scoring service. The cost is that you need that trained model.

Corrective RAG (CRAG) puts a lightweight evaluator after retrieval that grades the retrieved chunks as correct, incorrect or ambiguous. On a poor grade it takes corrective action — rewrites the query, or falls back to a broader source such as web search. This is the pattern we reach for most, because the evaluator is small, cheap and easy to reason about, and it fails safe.

Adaptive retrieval decides per query whether retrieval is needed at all. Worth it when a meaningful share of traffic is chit-chat or general knowledge that your corpus does not cover — you skip the retrieval cost and avoid dragging in irrelevant context.

You do not need to adopt these as branded architectures. In practice, most of the benefit comes from adding a retrieval evaluator and a single retry, which is a day of work rather than a rewrite.

Which pattern earns its latency depends on your corpus and your question mix, so we benchmark them against a client’s real questions before committing to one in a production RAG build.

What it costs

This is the part that gets underestimated, and it is why we do not recommend agentic retrieval by default.

Latency multiplies. Every extra hop is another retrieval plus another model call. A single-shot RAG answer might land in 800ms; an agentic one that retrieves twice and reflects once can take three to four seconds. For an interactive assistant that is the difference between fast and sluggish.

Cost becomes a distribution, not a number. A well-behaved query might use two model calls. A confused one loops until something stops it. Averages hide this — the long tail is where the spend is. Step caps and per-request token budgets are not optional here, they are what makes the bill predictable. Same discipline as any AI agent, for the same reason.

Debugging gets harder. When a fixed pipeline returns a bad answer, you check the retrieved chunks. When an agentic system does, you need the whole trajectory — which queries it issued, what each returned, how it graded them, why it stopped. Without trace-level observability you are guessing.

When to use it

Use agentic retrieval when questions genuinely require multiple hops, when your corpus is heterogeneous enough that one query shape does not fit, when users ask comparative or aggregative questions, or when a wrong answer is expensive enough to justify the reflection pass.

Stay with single-shot retrieval when questions map cleanly onto documents, latency is user-facing, or you have not yet measured retrieval quality. That last one matters most.

Fix retrieval before adding a loop

The most common mistake we see: a team adds agentic retrieval to compensate for retrieval that was never good enough, and gets a slower, more expensive system that is wrong in more elaborate ways.

Diagnose first. Take twenty questions the system answered badly and check whether the correct passage was in the retrieved set at all. If it was not, the problem is chunking, hybrid search or the embedding model — and no amount of looping fixes a retriever that cannot find the passage. Agentic retrieval multiplies whatever retrieval quality you already have; it does not create it.

Fix chunking, add BM25 alongside vector search, add reranking. Then, if multi-hop questions remain unanswered, add the loop.

Building it without a framework

Agentic RAG does not require adopting an agent framework. The minimal version is a loop with a grader:

Rewrite the user question into a search query.
Retrieve.
Grade the passages for relevance with a small, cheap model.
If the grade is poor, rewrite the query and retry — once, maybe twice, never unbounded.
Generate with citations, refusing if nothing survived the grade.

That is a structured tool-calling loop, roughly a hundred lines, and it captures most of the benefit. A framework earns its place when you need durable checkpointing and replay across many hops — LangGraph is our default there because state is explicit and a failed run resumes rather than restarts.

Evaluate the trajectory, not just the answer

Agentic systems fail in ways output-only testing misses: right answer reached by a wrong route, correct passages retrieved then ignored, a reflection pass that approves an unsupported claim.

Track how many hops a query took, how often the grader rejected the first retrieval, and how often the loop hit its cap without resolving. That last number is your real quality signal — a loop regularly exhausting its budget means retrieval is failing, not that the cap is too low. More on that in RAG evaluation.

The takeaway

Agentic RAG is a real improvement for multi-hop and comparative questions, and a real cost in latency, spend and debuggability. Add a retrieval evaluator and one retry before you add anything more elaborate, and only after you have measured that single-shot retrieval is genuinely the bottleneck.

Sources

EpochC provides custom RAG development services — hybrid retrieval over pgvector, Neo4j and BM25, with evaluation harnesses and grounding built in. See the graph + vector retrieval case study — +40% query accuracy — or start a project.

Agentic RAG: self-RAG, corrective RAG and when the loop is worth it