work / 02
A support assistant that answers with sources.
A client’s support team was answering the same questions twice — once in the product, once in Slack — while a generic chatbot would have invented confident answers. We built an assistant that answers only from the company’s own documentation and ticket history, and shows the source under every answer.
the problem
Documentation goes stale, tickets are where the real answers live, and customers ask in two places. A support answer that cannot be traced back to a source is worse than no answer: it costs trust every time it is wrong, and nobody notices until a customer does.
what we built
A retrieval pipeline over the client’s documentation and past tickets, an answer layer that cites the passage it used, and permission checks at query time so an answer never shows a customer something they are not allowed to see. It runs inside the web product and in Slack, on Python with the OpenAI API and a pgvector store — the model is one replaceable component, not the product.
the decision that kept it simple
Every prompt or retrieval change goes through a fixed evaluation set before it ships. That one rule makes the assistant improvable without guesswork: we can say what a change did to answer quality, instead of finding out from customers. It also keeps the system honest — an answer that loses its source loses its place.
where it stands
In production use, embedded in the client’s web app and Slack. The same pattern — retrieval, citations, permissions, evaluation — is what we now reuse when a product needs AI that can be trusted.