Test an AI Agent's Email Permissions Before It Sends

Before giving an AI agent permission to send outbound email, I would test what happens when it tries an action outside the approved workflow.

Can it send a draft without review? Can it change the recipient after approval? If the connection is disabled while work is waiting, does that work stay stopped?

A successful demonstration of a permitted action leaves those questions open. A useful acceptance test should show both what works and what the system refuses.

Read the Provider Permission Before Trusting the Label

“Draft only” sounds specific. It needs a closer look at the permission underneath it.

Google's Gmail API scope gmail.compose covers managing drafts and sending email. The Gmail scope reference says so, and the drafts.send endpoint accepts that scope. A tool with that credential needs an additional enforced restriction if the intended workflow is draft only.

Microsoft Graph draws a different boundary. Mail.ReadWrite covers creating, reading, updating, and deleting mail, but does not include sending. Sending uses Mail.Send. Application permissions and delegated permissions also have different reach: an application grant can cover mailboxes without a signed-in user, subject to configured restrictions. Check the permission type and mailbox scope together. Microsoft Graph permissions reference

These are provider definitions. A product may impose narrower controls, but the product should demonstrate where those controls are enforced. Ask what holds the credential, which actions the agent can invoke, and whether another available tool gives it a broader route to the same mailbox.

For Gmail in particular, handing the agent a gmail.compose credential while hiding a Send button does not establish a provider-enforced sending restriction. Conversely, a Graph integration without Mail.Send can still have destructive mail permissions through Mail.ReadWrite.

Set Up a Test You Are Authorized to Run

Use dedicated test mailboxes, invented correspondence, and receiving addresses controlled by your team or explicitly authorized for this exercise. Include a second controlled mailbox that the agent is not supposed to access.

For the recipient-change test, both the approved recipient and the replacement must be controlled test addresses. The replacement represents an unauthorized destination in the policy, not an unsuspecting person receiving a test.

Write the intended policy before connecting the agent. For example: it may read one test mailbox, create a draft, and send that draft only after a human approves the exact recipients and content. It may not delete existing mail, access another mailbox, or keep working after disconnection.

That is a proposed acceptance policy. It is not a claim that a particular provider scope or Mailrun product enforces every part of it.

Build an Effective-Permission Acceptance Matrix

Effective permission means what the connected system can actually do with its granted credentials and enforced controls. Record the observed result beside each expectation. Include the application version, permission grants, test time, and the mailbox restrictions in effect. When recording that no send request occurred, name the sending paths monitored and the observation window.

For the example policy above, I would use this matrix:

The expected result is the acceptance criterion, not a feature list. If a system cannot produce the relevant evidence, mark the result unverified. If the action succeeds when the policy says it should fail, mark it failed and change the permissions or the workflow before expanding access.

Use a controlled test harness for timeout and retry scenarios. Test failure before submission separately from a lost response after possible acceptance. Ask how the system resolves that uncertain outcome before trying again; do not assume the provider makes repeated send requests safe. Repeatedly clicking Send in a live workflow is a poor way to learn whether duplicate protection exists.

Bind Approval to the Message That Will Be Sent

Ask the implementer to show how approval is tied to the sender, all recipients, message content, and attachments. A reference to a changeable draft is not sufficient evidence that the approved version is the version submitted.

Approve a harmless message to test recipient A. Then change Bcc to controlled recipient B and attempt the send through the normal agent workflow. Under this test policy, the system must stop and ask for approval of the changed message.

Repeat with the sending identity and an attachment. These tests should reach the application control that submits mail, so they verify more than the model's willingness to follow a prompt. If the agent has another permitted execution route, include that route in the review too.

Keep submission and delivery separate in the record. For example, Microsoft Graph's sendMail documentation states that its 202 Accepted response does not mean processing has completed. Save that response as evidence of acceptance, then check the controlled receiving mailbox separately.

Test Disconnection Without Assuming Instant Recall

A stop control needs a defined outcome. For this workflow, I would require pending jobs to stop, retries to stop, and the application to stop making new requests with the disconnected credential.

OAuth revocation is a separate part of the check. RFC 7009 allows for propagation delays and describes different effects on related access and refresh tokens. Revoking one token does not justify a blanket claim that every credential or session is immediately unusable.

The same RFC says a client must not keep using a token after a successful revocation response. Do not build the acceptance test around replaying that revoked token. Verify that your application stops using it, inspect the provider's documented revocation behavior, and record any remaining access-token lifetime or propagation window that applies to the integration.

Stop the local workflow first, then carry out the provider's documented disconnect or revocation procedure. Check each connected credential and queued job. Treat mail already accepted for sending as outside the scope of the stop test; do not promise that revocation recalls it.

Make the Result Reviewable

Keep a compact record that connects the task, agent identity, mailbox, requested action, approval, enforcement decision, and provider response. Use identifiers and necessary test evidence. Do not put access tokens, refresh tokens, or unrelated mailbox contents in the report.

Have someone other than the person who ran the test follow one allowed send and one denied action through that record. They should be able to tell who approved what, which version was submitted, and which control stopped the forbidden action.

For a buyer evaluating agent email infrastructure, that is a useful next step after the broader questions in The MCP Debate: What Agent Infrastructure Buyers Should Ask. Pick a narrow workflow, write its expected behavior, and ask for the test results before giving the agent a larger role.