Before an AI Agent Replies, Check What Kind of Email Arrived

Imagine an AI agent sends an introduction and receives an out-of-office notice. It writes a polite acknowledgment. The other mailbox answers automatically again.

I would want that scenario in the test plan before allowing the agent to handle live replies. Good prose does not resolve the first decision: whether the incoming message calls for a response at all.

For outbound email, an arrival might be a human answer, a delivery report, an automated acknowledgment, or something the system cannot classify confidently. Give each a handling rule before asking a model to compose the next message.

Read the Automation Signal

The Auto-Submitted header provides a useful starting point. RFC 3834 defines auto-replied for automatic responses and auto-generated for automated messages that are not direct replies. A value of no can indicate human origin.

The same standard recommends that automatic responders not answer messages whose Auto-Submitted value is anything other than no. It also recommends marking an automatic response with Auto-Submitted: auto-replied. These are email protocol recommendations, not a claim that a particular AI product implements them. RFC 3834, Sections 2 and 3.1.7

For example, this synthetic excerpt should send your test down the automatic-response path:

From: Maya <maya@example.com>
Subject: Out of office
Auto-Submitted: auto-replied

The excerpt is not a complete message. Do not make the subject line the deciding factor, and do not treat a missing header or no as proof that a human wrote the email. Your policy needs an uncertain category as well.

If the integration exposes only a cleaned-up body, ask the implementer how the router obtains the original headers and message structure. Decide explicitly what happens when that evidence is unavailable. For a first deployment, I would hold uncertain cases for a person rather than automatically reply.

Route Delivery Reports to Delivery Handling

A delivery status notification, or DSN, is another kind of incoming mail that should not become a sales conversation. RFC 3464 defines a structured message/delivery-status part inside a multipart/report message. Its per-recipient Action field distinguishes outcomes including failed, delayed, and delivered. A delayed result means the reporting server is still trying; delivered does not establish that the person read the email. RFC 3464, Sections 2.1 and 2.3.3

I would route those reports to the component responsible for delivery state. Keep the report linked to the relevant outbound message and recipient. Before changing state, apply the integration's trust checks and corroborate the report against sending records. DSNs can be forged; a matching identifier alone is not authentication. Hold uncertain reports for a person. RFC 3464 discusses forgery explicitly.

Avoid treating the returned copy of your original email as new instructions. For example, a failure report may quote an introduction that ends with a question. That question still belongs to your original message; it is not a new request from the recipient.

The deeper diagnostic work belongs in the delivery workflow. How to Read SMTP Bounce Codes Like an Operator covers the evidence to keep when a send fails. Here, the immediate goal is to prevent the drafting step from handling a report as a reply.

Write a Routing Policy a Person Can Inspect

Start with a narrow policy for one controlled mailbox. Here is the policy I would use for a pilot, not a description of built-in Mailrun features:

An apparent human reply still needs the ordinary checks on sender identity, recipients, content, and approval before anything is sent.

Keep the received message available to the person reviewing a hold. A label such as “automatic” is more useful when the reviewer can see which header or report field supported it. Where a model helped classify the text, distinguish that judgment from directly parsed evidence.

Test the Cases That Should Produce No Reply

Use invented correspondence and mailboxes your team controls or has explicit permission to test. Have a controlled responder return the synthetic automatic-reply signal, then observe whether the agent records it without submitting a reply. Do not run an open-ended loop to see how long the system lasts.

Next, submit a delivery-report fixture with a quoted copy of the original outbound message. It should reach delivery handling without starting a fresh conversation. Include both delayed and failed outcomes, plus a malformed report that needs review. This checks the routing decision as well as the parser's successful case.

Test an ordinary human-written message too. Confirm that it reaches drafting and the expected approval step. A system that prevents loops by silently discarding every incoming email has not passed a useful acceptance test.

Finally, present the same controlled event twice. I would expect one handling record, with the repeated event reconciled against it. Choose the identifier and retention period with the implementer; do not assume a subject line is a unique key. If sending becomes uncertain, hold the case for reconciliation rather than turning a repeated event into another submission.

For each case, keep the message identifier, classification evidence, routing decision, and any provider request made during a defined observation window. Do not put access tokens or unrelated correspondence in the test report.

Put a Limit on the Conversation

If you later allow automatic sending, I would cap attempted reply submissions per conversation within a defined time window, including retries in that count. Choose the cap, the window, and how related messages are grouped for the workflow; no single number is universally safe. At the cap, stop submissions and put the case in a person's review queue. Include the behavior after a restart in the test; restarting a worker should not erase the reason a conversation was stopped.

Have the application owner show where the stop is enforced. Ask what happens to queued work and how an operator can take over an uncertain conversation. These are proposed operating controls that need demonstration, not assurances supplied by a prompt.

Before widening the pilot, read the record of one held message and one permitted draft from beginning to end. You should be able to explain why each reached its next step. That explanation is useful even when the agent's writing is already excellent.