Imagine your service team changes a setup requirement on Monday. The public guide now describes the new process, but an email agent still has an older instruction in its working context. A prospect asks a familiar question on Tuesday. The agent gives the answer that was correct last week.
The problem is easy to miss because the reply can sound reasonable and cite a real document. The useful question is whether that document still governs this customer's situation.
Once an agent is answering business email, changes to the business need a path into the workflow. Here is a practical way to manage one changed answer, from the information owner's decision through the next outgoing reply. These are proposed operating practices for your application and team, not a description of automatic Mailrun features.
Give Each Important Answer an Owner
Start with the sources that support answers people might act on: setup requirements, service coverage, eligibility, support arrangements and commitments. For each, name a person who can resolve uncertainty about what the business currently offers.
The owner need not review every routine message. Their job is to decide which information is authoritative, when a change takes effect, and which customers or circumstances it covers.
Keep a small source register. A spreadsheet is enough for an initial workflow:
| Record | What to write down |
|---|---|
| Approved source | The specific document or record the agent may use |
| Information owner | The person who can approve or resolve a change |
| Scope | The audience, service and circumstances the answer applies to |
| Version and effective date | Which version governs, and when it takes effect |
| Places used | Instructions, retrieval collections, templates or other known copies |
| Recheck trigger | A scheduled review or a business event that requires one |
This is an application of a broader principle. NIST's AI Risk Management Framework calls for clear responsibilities, ongoing review and change management across the AI system's life. It does not prescribe this register or a universal review interval. Choose the interval according to how often the information changes and what a wrong answer could cause.
Resolve Conflicts Before Asking the Agent to Choose
Suppose the revised guide says a specialist must confirm an installation date, while an older sales sheet suggests the date can be agreed during the first inquiry. The agent should not settle that disagreement by picking the newest-looking filename or the more confident paragraph.
Ask the information owner to record the decision: which source applies, which statement is retired, and whether existing customers have different arrangements. Until that decision is available, route questions about installation commitments to the responsible person.
Separate new information from new authority. A customer's email can describe their circumstances, but it should not silently rewrite your approved service terms. Likewise, a retrieved document can contain instructions aimed at the model. OWASP's prompt-injection guidance describes attacks through external content and recommends separating untrusted material and limiting privileges. A source register is useful bookkeeping; it is not a security control that makes every document safe.
Follow the Change Into Prepared Work
Replacing the source is only one part of the review. Inspect the places your own application actually uses: a copied instruction, a retrieval collection, a saved conversation, a draft waiting for approval, or a task prepared earlier.
Do not assume that editing the original document also updates those places. Ask the application owner which copies exist and how updates become available. If the system cannot show that an affected draft uses the approved answer, hold that draft for a person to check before sending it.
For the hypothetical installation change, make a short change record:
- The old statement promised that a date could be agreed during the first inquiry.
- The approved statement requires specialist confirmation for the specified service.
- The owner identifies when it takes effect and any exceptions.
- Someone checks pending replies that discuss dates, then records what was corrected or held.
Use the workflow's actual send or approval controls to hold affected work. An instruction telling the model to be careful is not evidence that a prepared message cannot leave the queue. If the change reveals a reply that has already gone out, have the information owner decide whether a correction is needed and who should send it.
Rehearse the Changed Question
Use controlled test correspondence to ask the same question before and after the update. Record the source version available to the application, the proposed answer, and the reviewer's decision. Where the system exposes retrieval evidence, inspect it; do not treat a model's claimed citation as proof of what it actually used.
Include a follow-up that quotes the old answer. A useful result should acknowledge the discrepancy and use the approved handling for that situation, rather than treating the quoted message as a new business instruction. Also test a question outside the source's scope. An agent that cannot establish the answer should have a workable route to the named person.
Keep test material free of unnecessary customer data. A clean test can help expose a problem, but one successful reply does not establish that every future answer will be correct. Repeat the relevant checks when sources, connected tools or the application change.
Start With One Answer That Matters
Choose one question whose answer your business has changed recently. Identify the owner, the governing source, and a pending or test reply that would reveal whether the change reached the workflow.
The result you want is concrete: a colleague can explain why today's answer applies, where the obsolete answer was used, and what happens when the sources disagree. That gives the team a useful maintenance habit as the agent takes on more conversations.
Talk through your Mailrun setup.
Ask about Agent Identities for your agent or ours, or Mailrun Core for your sequencer. Our team can help you choose.
Book a call