LLM consulting: what to buy and what to skip
Short answer: most useful LLM consulting is short and diagnostic. Two weeks measuring whether your retrieval actually finds the right passages, whether your evaluation set exists, and whether the failure path is designed. Frequently the output is a list your own engineers can apply. Buy a build only after that diagnosis, and only if it says one is needed.
What LLM consulting should cover
Retrieval quality first. For a representative set of real questions, is the answer-bearing passage actually being retrieved. That single measurement reframes most projects, because it separates a retrieval problem from a model problem and they have completely different fixes.
Evaluation. A held-out set of real inputs with known correct outputs, scored on every change, with retrieval measured separately from generation. Without it, every later adjustment is a guess.
Grounding architecture. Whether answers are assembled from retrieved passages with sources attached, or whether the system is asked in a prompt not to invent things. Only the first is a control.
Guardrails and cost. What the system is permitted to do, enforced in code. Per-run token ceilings, because a looping system bills. Model routing, which is usually the largest saving available in a production deployment.
If an engagement does not touch those four, it is strategy consulting with LLM vocabulary.
When you do not need it
Your problem is individual productivity. Drafting, summarising, exploring. Buy the product and spend the budget elsewhere.
You have not defined the task. No consultant can fix an unclear objective, and the discovery will cost you more than writing it down yourself would have.
Nobody can say which outputs are correct. You need a domain expert available before anyone starts. Their availability sets the timeline more than any engineering does.
What LLM companies differ on
The vocabulary is cheap and the positioning is easy to write, so judge on evidence.
Ask what is running in production and who operates it. Ask how they measure a change. Ask what happens to a low-confidence output. Ask when they would tell you to buy a product instead.
Our own search data makes the point about positioning: one specialist firm that reads impressively on its site ranks for 35 keywords with none in the top twenty. Marketing depth and engineering depth are not the same thing, and neither is visible from a homepage.
What a good engagement looks like
Fixed price, two to three weeks, diagnostic before build.
Week one: the corpus, the real questions, and an evaluation set agreed with you. Week two: retrieval measured in isolation, chunking and ingestion reviewed against the document types you actually hold, the permission model checked, the failure path examined. Output: a prioritised list of what to change and what it is worth.
That is how our custom RAG development services engagements start, including the ones that end there because the fix turned out to be small.
Where the money actually goes in a build
Not the model. Roughly a third into retrieval and corpus work, a third into integration with your systems of record, and a quarter into guardrails and evaluation. Prompt and model work is the remainder.
Any proposal that inverts those proportions is describing something that will demo well and struggle in production.
What you should receive
A consulting engagement that produces only a slide deck has failed, and this is the most common failure in the category.
The output of a useful engagement is four artefacts. A written assessment of what is feasible on your actual data, with the accuracy ceiling estimated rather than asserted. An evaluation set built from your real cases, which is the thing that lets anyone measure progress afterwards. An architecture that names the constraints it was designed around: where data may go, what has to be explainable, what latency budget applies. And the arithmetic: volume, time saved, build cost, running cost, and whether it clears.
That last one is where an honest engagement most often recommends against the build, and it is the reason a fixed-price diagnostic is better value than a longer discovery inside a build contract.
How to structure the engagement
Two weeks, fixed price, with the deliverables named in the statement of work. Long enough to touch real data, short enough that a wrong hire costs you little.
Insist on access to real data within the first three days. A consultant working from sample files or a description of your data is producing an opinion, and the gap between described data and actual data is where most projects go wrong.
Agree in advance who owns what is produced. The evaluation set in particular should be yours, in your repository, because it outlives the engagement and it is the artefact a subsequent supplier will need.
Questions worth asking before you sign
- What is running in production from your previous engagements, and who operates it now?
- Show us an evaluation set you have built for a client. Not the results, the structure.
- When did you last tell a client not to build something, and what did you recommend instead?
- Who does the work, by name, and are they the people in this meeting?
- What would make you say this project should not proceed?
The last question is the one that separates advisers from vendors. A consultant with no answer has never turned work down.
The takeaway
Buy LLM consulting as diagnosis, not as a build contract. Two weeks measuring retrieval, evaluation and the failure path tells you whether you need a project at all, and a consultant unwilling to sell that is a consultant selling a project you may not need.
EpochC provides generative AI development services and custom RAG development services, starting with a fixed-price diagnostic. See the graph and vector retrieval case study or start a project.