Building a support assistant that quotes your real policies
A useful ecommerce support assistant retrieves your real policy text and quotes it, rather than generating an answer from a general model. Ground every response in a source document, make it decline anything it cannot support, and route those declines to a person. An assistant that guesses at returns terms creates commitments you are legally held to.
Support assistants are the most requested automation on our AI enquiries and the one with the widest gap between a good build and a bad one. The difference is not model quality. It is whether the assistant is allowed to answer from what it knows, or restricted to answering from what you wrote.
The failure that makes this risky
Ask a general model what your returns policy is and it will tell you. Confidently, in your tone of voice, with a plausible number of days that came from the general shape of ecommerce rather than from your terms.
The customer has no way to tell the difference. They now believe they have thirty days because your website said so, and the conversation about whether your website really said so is one you will lose in every sense that matters.
Under the Consumer Protection from Unfair Trading Regulations, what a business tells a consumer about their rights matters. We are not lawyers and this is not legal advice — the engineering conclusion is simply that an invented answer is not a harmless bug, and the architecture has to make inventing impossible rather than unlikely.
Grounding, in plain terms
Grounding means the assistant does not answer from memory. It retrieves the relevant passage from your documents, and its reply is built from that passage or it does not reply at all.
The mechanism is unglamorous. Your policies, delivery terms, sizing guidance and FAQ answers are indexed. A customer question retrieves the closest passages. The model’s instruction is to answer using only those passages and to decline if they do not contain the answer.
Two details decide whether it works.
Quote rather than paraphrase where it counts. For anything with a number in it — a returns window, a delivery cost, a warranty term — the reply should carry your wording, not a summary of it. Paraphrase is where numbers drift.
Cite what it used. A reply that links the policy page it answered from lets the customer verify and lets you audit. It also makes wrong answers debuggable, because you can see which document it read.
Scope: what an assistant should and should not answer
Scope is a design decision, not a capability question, and it is where most of the risk is managed.
| Question type | In scope? | Because |
|---|---|---|
| Returns, delivery, warranty terms | Yes | The source of truth is a document you control |
| Sizing and fit guidance you published | Yes | Same, provided the guidance actually exists |
| Stock availability | Yes | Read from the store, not from the model |
| Order status | Only authenticated | Personal data, and identity must be established first |
| Product specifications | Only from structured data | Prose descriptions are where invention happens |
| Whether a product suits a use case | Judgement call | Fine as opinion, dangerous as fact |
| Anything about a competitor | No | No source of truth, and nothing to gain |
| Discounts or exceptions | No | An assistant that can give a discount will give a discount |
The last row is not hypothetical. An assistant with no rule against it will agree to a goodwill gesture if a customer pushes, because agreeing is what the conversation seems to want.
Order status deserves its own paragraph
Order status is the most requested capability and the one that turns a support widget into a data-protection question. The rule is simple and non-negotiable: no order information without authentication.
An assistant that will look up an order from an order number and an email address will do it for anyone who has both, and both appear in forwarded confirmation emails. Read the order from the order system after the customer has signed in, or do not offer the feature. ICO guidance for organisations is the place to start on what your obligations actually are.
The refusal path is the product
Most builds treat “I don’t know” as a failure state. It is the opposite: the refusal is what makes every other answer trustworthy.
A refusal that works has three parts. It says plainly that it cannot answer rather than producing something vague. It offers the handover immediately rather than after three more attempts. It carries the conversation to the person taking over, so the customer does not start again.
That last part is where assistants most often disappoint. A handover that drops context makes the customer explain twice, which is worse than not having offered help — and it is entirely a plumbing problem rather than an AI one.
What the logs are actually for
Every question the assistant declines is a gap in your documentation, written in a customer’s own words. That log is the most accurate content brief you will ever get.
In the deployments we have run, the decline log reliably contains the same handful of questions repeated dozens of times — questions the policy pages genuinely do not answer. Writing those answers improves the assistant, the policy pages, and the human support queue at the same time, and the assistant paid for itself by finding them.
Log the answered questions too, with the passage retrieved. When somebody eventually reports a wrong answer, the only way to fix it is to see what it read.
Testing it before customers do
Three test sets, and the second is the one people skip.
Questions it should answer. Fifty real questions from your support inbox, with the correct answers written down beforehand. Anything wrong is a retrieval problem, not a model problem.
Questions it should refuse. Legal advice, competitor comparisons, discount requests, anything about another customer’s order, and questions whose answer is not in any document. A pass here means refusing cleanly, not answering carefully.
Questions designed to break the rules. Polite persistence, hypotheticals, instructions embedded in the question, and requests to ignore previous instructions. The OWASP Top 10 for LLM applications is the useful reference for what to try, and prompt injection is the entry worth reading twice.
Re-run all three whenever the model, the prompt or the documents change. Behaviour drifts on model updates, and the only way to know is to test the same questions again.
Where automation genuinely helps here
Setting the risks out at length makes this sound discouraging, which is not the conclusion.
A grounded assistant answering delivery, returns and sizing questions removes a large share of a support queue’s volume, and it removes the most repetitive share. It answers at three in the morning. It is consistent, which humans under pressure are not. It never gets tired of the same question on its fortieth repetition.
Those are real gains, available today, at a scale that matters for a small team. They arrive on the condition that the assistant is restricted to what you actually wrote — which is a build decision made at the start, not a setting turned on later.
What we build, and what we decline to build
We build grounded assistants scoped to policy, delivery, returns and sizing, with an audited refusal path and a human handover that carries context. We instrument the decline log from day one because it is the most valuable output.
We decline to build assistants that answer from a general model without retrieval, that discuss orders without authentication, or that are permitted to negotiate. Those are not harder to build. They are easier, which is exactly the problem.
