◆ Case study · EdgeCIO · FinTech
We combined a knowledge graph with vector search and lifted query accuracy 40%
A dual-module retrieval platform combining Neo4j knowledge graphs with MongoDB vector embeddings, orchestrated with LangChain and served over gRPC microservices with Redis caching.
+40%
query accuracy
+35%
answer relevance
<200ms
end-to-end latency
The problem
Pure vector retrieval was hitting a ceiling. Embeddings are very good at finding text that means something similar to the question, and structurally unable to answer questions about how entities relate to each other. Anything requiring a hop — this entity, then the thing connected to it — came back with passages that read relevantly and did not contain the answer.
The failure was hard to see from accuracy numbers alone, because the retrieved chunks were topically correct. They were simply not the chunks that resolved the relationship the question asked about.
The platform also had a latency budget. Retrieval sat in an interactive path, so any accuracy gain that cost a second of response time would not have been worth taking.
What we built
-
Model the relationships explicitly
We built a Neo4j knowledge graph over the entities and the connections between them, so relationship questions get answered by graph traversal rather than by hoping the right sentence landed in the same embedding neighbourhood. Relationships that were previously implicit in prose became queryable structure.
-
Keep vector search for what it is good at
Semantic search over MongoDB embeddings stayed, because it remains the right tool for open-ended and descriptive questions where the user’s phrasing will not match the source vocabulary. The insight was never that graphs beat vectors — it is that they fail on different questions.
-
Run both modules and merge
The two retrieval modules run as a dual system orchestrated with LangChain, and their results are combined before the answer is composed. Structured facts from the graph anchor the entities and their relationships; retrieved passages supply the surrounding context and language. That combination is where the +40% query accuracy and +35% answer relevance came from.
-
Serve it over gRPC
The retrieval modules are gRPC microservices, which keeps serialisation cheap and the service boundaries typed. In an interactive path where two retrieval systems run per query, per-hop overhead is not a rounding error.
-
Cache the repeated work
Redis caches hot traversals and embedding lookups. Real query distributions are heavily skewed — a small set of entities accounts for a large share of traffic — and caching that head is what kept end-to-end latency under 200ms while running two retrieval systems instead of one.
The outcome
- +40% query accuracy against the vector-only baseline.
- +35% answer relevance, judged on the same evaluation set.
- Under 200ms end-to-end latency, with both retrieval modules in the path.
- Relationship questions became answerable at all, rather than returning topically-close passages that missed the point.
Why this stack
- Neo4j
- Native graph traversal. Multi-hop relationship queries stay fast where relational joins degrade.
- MongoDB
- Vector embeddings alongside the document store, avoiding a separate vector database to synchronise.
- LangChain
- Orchestrates the dual-module retrieval and result merging.
- gRPC
- Typed service boundaries and cheap serialisation, which matters when two retrieval systems run per query.
- Redis
- Caches hot traversals and embeddings. Query distributions are skewed, so caching the head pays for itself immediately.
Frequently asked
When is graph RAG worth the extra complexity over plain vector RAG?
When your questions are about relationships rather than topics. If users ask “what does this document say about X”, vector search is sufficient and a graph is overhead. If they ask “which suppliers connect to this entity through a subsidiary”, vector search structurally cannot answer it — the answer is not contained in any single passage. Auditing your actual failed queries tells you which situation you are in.
Does running two retrieval systems double your latency?
Not if they run concurrently and you cache the head of the distribution. This system stayed under 200ms end to end with both modules in the path. The cost is real but it lands in infrastructure complexity, not in response time.
Do you have to rebuild the graph when the data changes?
No. The graph is updated incrementally as entities and relationships change. A full rebuild is a recovery procedure, not part of normal operation.
Could you use pgvector instead of MongoDB here?
Yes, and on other builds we do. The choice follows where the source data already lives — adding a second datastore purely for embeddings creates a synchronisation problem that is usually worse than any performance difference between the options.
More on rag & retrieval
- GuideEnterprise RAG architecture: permissions, freshness and evaluation
- ServiceRAG development & retrieval systems
- ArticleHow to stop your RAG system from hallucinating
- ArticleRAG vs fine-tuning: which do you actually need?
- ArticleKnowledge graphs vs vector search for RAG
- ArticleRAG as a service vs custom RAG development
- ArticleChoosing an embedding model for RAG
- ArticleAgentic RAG: self-RAG, corrective RAG and when the loop is worth it
- ArticleRAG chunking strategies: semantic, fixed and structure-aware
- ArticleRAG evaluation: metrics, Ragas and the golden set
- ArticleRAG vs CAG: when to cache the whole corpus
- ArticleMultimodal RAG: retrieval over PDFs, tables and diagrams
- ArticleOn-premise RAG: retrieval that never leaves your network
- ArticleRAG consulting: what to ask when your system already exists
- ArticleRAG for life sciences: provenance and validated retrieval
Have a project like this?
Production-grade, owned end to end. Usually a reply within a day.