Conversational AI Design: What Works in Production

Short answer: the craft changed shape. Designing intents, entities and dialogue trees was about anticipating every path. With an LLM you no longer control the wording, so the design work moves to boundaries — what the assistant may say, what it must never claim, where it stops and hands over, and how it behaves when it does not know.

Most conversational design material still assumes the old model. Here is what carries over and what does not.

What the work actually consists of now

Conversational AI design used to mean drawing flows. It now means writing down four things before anyone builds, because each is a decision the model cannot make for you.

The job. One sentence naming what a user is trying to finish. “Find out whether a claim was paid” is a job. “Improve customer engagement” is not, and a project that starts there produces an assistant nobody can evaluate.

The boundary. What the assistant may assert, and what it must never assert without a source. This is the single highest-value artefact in the whole process, and it is usually a list of twenty lines. In one clinical build the boundary was short and absolute: never state a dosage, never interpret a result, never speculate about a diagnosis. Everything else in the system followed from it.

The repair path. What happens when retrieval finds nothing, the user is ambiguous, or a tool call fails. Covered below, because it is the part everyone skips.

The exit. How a conversation ends, in both directions: resolved, or handed to a person with context attached.

Notice that none of these are wording. The model handles wording, and handles it better than a script did. What it cannot do is decide what is true, what is permitted and when to stop.

A worked example of the difference

A user asks an insurance assistant: “why hasn’t my claim gone through?”

A dialogue-tree system matches this to a claim_status intent, extracts no entities, and asks for a claim number. If the user does not have one to hand, the conversation ends there.

A grounded assistant looks up the claims attached to the authenticated user, finds one pending, retrieves the reason code, and answers:

Your claim from 3 September is on hold because the provider’s invoice is missing a diagnosis code. We have requested it from them. Nothing is needed from you right now. Source: claim #48213, status updated 9 September.

Three design decisions produced that, and none of them are phrasing. Retrieval is scoped to the requesting user, so the assistant can look up a claim without being given a number. The answer cites the record it came from, so a support agent can verify it. And it says explicitly that no action is needed, because the actual question underneath “why hasn’t it gone through” is almost always “what do I have to do about it”.

The failure mode worth noticing: a model with no retrieval will answer that question fluently and completely wrongly, and it will sound exactly like the good answer.

What the dialogue tree was actually for

Intent classification and scripted responses existed because the system could not compose language. Every path was enumerated because every path had to be.

That produced two chronic failures. Anything outside the enumerated intents hit a fallback, and users learned quickly that the assistant only understood a narrow phrasebook. And the trees grew unmaintainable — a mature bot became hundreds of intents nobody could reason about, where adding one risked shadowing another.

Generation removed both problems and introduced a different one: the system can now say anything, including things that are wrong. The design job inverted, from specifying what it says to constraining what it may say.

Grounding is a design decision, not an implementation detail

The most common production failure is confident invention, and it is a design problem before it is an engineering one.

“Only answer from the provided context” in a system prompt is a request, not a constraint, and it will be ignored under pressure. What works is structural: retrieve first, and if nothing comes back above a relevance threshold, the assistant says it does not know and offers a route forward rather than generating.

Designing that means deciding, deliberately: what is this assistant allowed to assert? What must always carry a citation? What questions should it refuse even when it could produce a plausible answer? Those are conversation-design questions with engineering consequences, and they belong in the design phase rather than being discovered in review. The mechanics are in how to stop RAG hallucinations.

Design the not-knowing path first

Teams design the happy path and treat failure as an edge case. Users experience the failure path constantly, and it decides whether they come back.

Three states need explicit design, not a shrug:

Does not know. Say so plainly, offer the next step. Never bluff, never pad.
Knows but must not say. Policy, permissions, regulatory limits. The refusal should explain that a limit exists without lecturing.
Partially knows. The hardest one, and the most common. Answering the part it can while being explicit about the part it cannot is genuinely hard to word well, and worth writing by hand rather than leaving to the model.

Getting these right does more for perceived quality than any improvement to the happy path.

Handoff is a feature

The moment someone wants a human, the job is getting them one fast with context attached. An assistant that fights the handoff destroys more goodwill than it ever saved.

Design it explicitly: an escape hatch available on every turn, the full conversation passed to the agent so nothing is repeated, and escalation triggered by detected frustration rather than only by explicit request. Making a user restate their problem after a failed bot conversation is the fastest way to make the whole programme unpopular internally.

Persona is smaller than people think

Persona work absorbs a lot of design time and returns little. Users are not looking for a personality; they are looking for an answer.

What actually matters is register and length consistency, and the discipline to keep both narrow. A support assistant that is chatty in one turn and terse in the next reads as broken regardless of how well-defined the persona document was. Pick a register, write four or five exemplars, and enforce them — that is most of the value.

Where persona genuinely matters is the failure states above. Those are the moments users judge, and generic apologetic filler is worse than a plain sentence.

Conversation length is a warning sign, not a goal

Engagement metrics flatter conversational systems badly. Long conversations usually mean the user is not getting what they asked for.

Design for shortest-path resolution. If the assistant can answer in one turn, it should — not ask a clarifying question it does not need. Clarifying questions are a real tool, but each one is a chance for the user to leave, so ask only when the answer genuinely branches.

Measure resolution, not turns, and treat the list of questions where retrieval returned nothing as your content roadmap. That list is the most actionable artefact a chatbot produces and most teams never look at it. We went deeper on the metrics in customer service chatbots.

Repair: the four ways a turn goes wrong

Every conversation that fails in production fails in one of four ways, and each needs a designed response rather than a generic apology.

Nothing was retrieved. The corpus does not contain the answer. Say so plainly, say what you can see, and offer the exit. “I do not have anything on cancellation terms for business accounts. I can put you through to someone who does.”

Something was retrieved but it does not answer the question. Harder, and more dangerous, because the model will compose over it happily. This needs a relevance check between retrieval and generation, not a better prompt.

The question was ambiguous. Ask exactly one clarifying question, and make it a choice rather than an open prompt. “Do you mean the trial that ended in July, or the current subscription?” beats “Could you give me more detail?”, which reads as the system stalling.

A tool call failed. Never surface the error. Say what did not happen and what happens next: “I could not reach the billing system just now, so I have not cancelled anything. Try again in a few minutes, or I can pass this to support.”

The pattern across all four is the same. Name the limit, state what is true, offer a route forward. What breaks trust is not an assistant that cannot do something, it is one that pretends.

What to write down before building

A workable conversational design deliverable is short:

The boundary list. What it may assert, what needs a citation, what it must refuse.
The three failure scripts. Does not know, may not say, partially knows.
Escalation rules. What triggers handoff, what goes with it.
Register exemplars. Four or five sample exchanges showing tone and length.
The evaluation set. Real user questions with expected behaviour, including questions it should decline.

That is more useful than a hundred-node dialogue tree and it survives contact with a model that writes its own sentences.

The takeaway

Stop designing paths and start designing boundaries. The assistant will produce the wording; your job is deciding what it may claim, where it stops, and how it behaves when it does not know. Design the failure states first — that is where users form their opinion.

EpochC builds AI chatbots and domain-grounded assistants on production RAG, with AI agents where the assistant needs to act. See four sub-agents behind one conversational API, or start a project.

Conversational AI design: the principles that survive production