RAG for contract analysis: what retrieval gets right and wrong
Short answer: contract analysis breaks the assumptions naive retrieval is built on. A clause means what a definitions section elsewhere in the document says it means, an amendment three documents away may have replaced it, and the most important finding is often that a clause is absent — which no retriever can return. RAG for contract analysis works, but only with structure-aware chunking, a document set that models the amendment chain, and an explicit checklist pass for absence.
Why contracts break naive retrieval
Four properties of contract analysis, each of which defeats a straightforward vector search over chunks.
Definitions live somewhere else. A liability clause turns on what “Losses” means, and that is defined in section 1 of a forty-page agreement. Retrieve the liability clause alone and you have retrieved a sentence whose meaning is not in it.
Cross-references are structural. “Subject to Section 8.3” is load-bearing. A chunk containing that phrase without Section 8.3 is worse than no answer, because it reads as complete.
Amendments override. The operative text is the master agreement as amended, and the amendment is a separate document that says “Section 4.2 is deleted and replaced with the following”. Retrieval over a corpus that treats both as equal peers will happily return superseded language.
Absence is a finding. The most valuable question in contract review is frequently “does this agreement have a limitation of liability clause?” and the answer “no” is what you needed. A retriever asked for something not present returns the nearest thing that is, which is precisely the wrong behaviour.
That last one deserves emphasis because it is the failure teams do not anticipate. Retrieval systems are built to always return something.
Chunk on structure, not on length
The default chunking strategy — fixed token windows with overlap — is close to the worst possible choice for contract analysis.
Contracts are hierarchical: parts, sections, subsections, clauses, schedules. That hierarchy is the meaning. Split on it instead, so a chunk is a clause or a subsection, and carry the path with it: which agreement, which section, which subsection, and the heading text at each level.
Then, on retrieval, expand. A clause chunk brings its definitional dependencies and any section it cross-references into the context alongside it. This costs context window and it is the difference between an answer that is correct and one that is plausible.
The general principles are in RAG chunking strategies; contracts are the case where structure-aware splitting stops being an optimisation and becomes a requirement.
Model the document set, not just the documents
A contract is rarely one file. It is a master agreement, an order form, several amendments, a schedule of rates, and sometimes a side letter, executed at different times and related in a way the files themselves do not state.
Before retrieval quality matters at all, the system has to know which documents belong to which relationship and which version of a clause is operative. That is an entity resolution and metadata problem, and it is where most of the engineering effort on a contract analysis build actually goes, well ahead of anything a model does.
Practically, each document carries its parties, its execution date, its type, and its relationship to others in the set. An amendment records what it amends. Retrieval then filters to the operative set and, where a clause has been superseded, returns the replacement with the history available rather than the original.
Get this wrong and every downstream accuracy number is measuring the wrong corpus.
Handle absence with a checklist, not a query
Since retrieval cannot return what is not there, contract analysis should not ask it to.
The pattern that works is a clause taxonomy: the set of provisions you care about, defined in advance. Limitation of liability, indemnity, termination for convenience, assignment, governing law, auto-renewal, exclusivity, whatever your review actually covers.
For each item, run a classification pass over the structured document rather than a retrieval query. Present or absent, and where present, which chunk. The output is a completed checklist with citations, and an absent item is a first-class result rather than an empty search.
This also gives you something retrieval alone never produces: a consistent, comparable output across a portfolio of agreements. That is the thing procurement and legal operations teams actually want, and it is why a clause taxonomy is usually the first deliverable rather than a chat interface.
Extraction and retrieval are different jobs
Two distinct requirements get bundled under contract analysis, and they want different machinery.
Extraction pulls specified values into structured fields: parties, effective date, term, renewal notice period, payment terms, cap on liability. Deterministic output, gradeable against a labelled sample, and it belongs to the same discipline as intelligent document processing services.
Retrieval answers open questions across a corpus: which of our agreements permit assignment without consent, what does this indemnity actually cover.
Build the extraction layer first. It produces the metadata that makes retrieval useful — you cannot filter to agreements expiring this year unless somebody extracted the expiry date — and it is the part with a defensible accuracy number.
What to measure
Contract analysis has a specific evaluation shape, and a generic RAG benchmark will not tell you what you need.
Clause-level recall on a labelled set. For each clause type in your taxonomy, on a sample of real agreements reviewed by a lawyer, does the system find it when present. Measure per clause type, because performance varies enormously between a well-formed governing law clause and an obligation buried in a schedule.
False-absence rate. How often the system reports a clause missing when it is present. This is the dangerous error and it should be tracked separately from ordinary misses.
Citation correctness. Does every assertion point at the specific clause that supports it, in the operative document. An answer with a citation to a superseded amendment is wrong in a way that looks right.
Definitional completeness. On a sample of answers turning on a defined term, did the definition make it into the context.
The broader methodology is in RAG evaluation. The contract-specific addition is that recall matters more than precision here, because a reviewer can dismiss a false positive in seconds and will never discover a false negative.
Where this pays and where it does not
Contract analysis pays on volume and repetition. A portfolio review across hundreds of agreements, diligence in a transaction, working out exposure across a supplier base when a term changes. Work that is currently done by reading every document because there is no other way.
It pays less on a single high-stakes negotiation, where a lawyer is going to read the document regardless and the system saves minutes rather than weeks.
And it does not replace legal judgement. The output is a structured, cited, reviewable summary that makes a lawyer faster. Anything presented as a conclusion rather than as evidence with citations is being sold beyond what the technology supports, and the liability question that follows is not one a vendor gets to answer.
The takeaway
RAG for contract analysis works when the system respects what a contract is: a hierarchy of clauses whose meaning depends on definitions elsewhere, amended by other documents, where absence is a finding. Chunk on structure, model the amendment chain, extract before you retrieve, and run a clause checklist rather than hoping a query surfaces a gap. Measure recall per clause type against lawyer-labelled samples, and treat false absence as the error that matters.
EpochC builds custom RAG development services over contracts and clinical, financial and technical corpora, with the intelligent document processing services that structure them first. See the graph and vector retrieval case study, or start a project.