Blog

How to limit AI agent data access: control what your agents can see

Aperisa ยท 7 min read

Checked on 2026-10-29.

Google's description of the read-only Gmail scope is six words long: "View your email messages and settings." Not the messages from one client. Not this month's. The agent can't send or delete with it, and it can still open any message in the mailbox.

To limit AI agent data access, you have to go below the OAuth scope. Decide three things per agent: which data areas it can reach, whether it sees work or personal data, and how much detail it gets. Then give each agent its own key so you can change or revoke one without touching the rest.

Our view: read-only is the floor, not the fix. It stops an agent from acting on your data. It does nothing about how much of your data the agent reads, and an agent's reads all go into its context, which is the part that can leak.

Why do AI agents usually get all of it or none of it?

Because OAuth scopes are cut per service and per action, not per question.

Google's Gmail API scopes page (last updated 2026-09-10) lists gmail.readonly under restricted scopes with the description "View your email messages and settings." Google describes restricted scopes as ones that "provide wide access to Google user data." The narrower read option on that page is gmail.metadata: "View your email message metadata such as labels and headers, but not the email body." That is a real level-of-detail split, but outside add-on scopes it is the only narrower read option, and an agent that needs to read an invoice can't work from headers.

Drive looks similar. The Drive API scopes page (last updated 2026-09-03) describes drive.readonly as "View and download all your Drive files." There is a narrow option, drive.file, which covers files "that you open with an app or that the user shares with an app while using the Google Picker API or the app's file picker." That works when a person picks each file. An agent that searches your Drive to answer a question doesn't have a person picking files, so it ends up on a broad scope.

Google's own advice on both pages is to "choose the most narrowly focused scope possible." For an agent, the narrowest useful scope is still the whole mailbox or the whole Drive. So if you want an agent that sees only invoices from one vendor, or only your work calendar, OAuth won't give you that. Something between the token and the agent has to.

That layer is usually an MCP server or a small proxy. It holds the Google token and gives each agent a narrower key. If you are new to that piece, what an MCP server is covers it.

Three questions to ask before you give an agent a key

Which data areas does this job need?

Name them. Email, documents, calendar, contacts, bank transactions. A bookkeeping agent needs transactions and invoices. It doesn't need your calendar or your contacts. A scheduling agent needs the calendar and maybe contacts. It doesn't need bank data at all.

If you can't say why the agent needs an area for the job you gave it, leave it off. You can add it later when a real request fails. Adding access is easy. Finding out what an agent read last week is not.

Work or personal?

Most people's accounts mix both. A personal Gmail gets client mail. A work Drive holds a scan of a passport from the visa application. An agent you set up to triage client email should not be reading your medical portal notifications because they happen to sit in the same inbox.

OAuth can't make this cut for you, since Google sees one account.

How much detail does the answer need?

Many agent jobs need a number, not the records behind it. "How much did we spend on software last quarter?" needs a total. It doesn't need every card transaction with merchant names and dates. "Do I have anything Tuesday afternoon?" needs free or busy. It doesn't need the attendee list and the meeting notes.

Giving an agent totals instead of full records is the least-privilege idea applied to detail. The agent can still do its job, and there is far less in its context to leak if a prompt injection in some email or web page gets it to repeat what it has seen.

Patterns that work in any harness

These don't depend on which tool you run. They apply to OpenClaw, Hermes, Claude Code, n8n or your own code.

One key per agent

Give every agent its own key, even if two agents use the same data source. A shared key has three problems. You can't scope it to one job, because it has to cover both. You can't tell from logs which agent made a request. And you can't cut one agent off without breaking the other.

Where the key lives matters too. Keep it out of config files and their backups. MCP API key config security goes through where each harness can read a key from.

Separate keys for agents that post in public

An agent that can post to social media, send email or reply in a shared channel can repeat what it read. Treat that agent as the riskiest one you run, and give it the smallest key. Often it needs no private data at all. If it does, keep it to the areas and detail level the post needs, and keep it on its own key so the narrow limits don't get loosened for some other agent's sake.

The same goes for any agent that reads content other people control, like inbound email or web pages. Keep its reading tools apart from its sending tools where you can. OpenClaw Gmail read-only shows one way to set that up for an inbox.

Revoke on change

Revoke a key when anything about the agent changes: a new job, a new model, a new skill or plugin, a teammate leaving, or the agent being retired. Then issue a fresh one with limits that fit the new setup.

This only works if revoking one key is cheap. With one key per agent, it is. With a shared key or a raw Google token in three configs, revoking means rebuilding, so people put it off.

What this doesn't fix

Limiting what an agent can see reduces what can leak. It doesn't stop prompt injection. An agent with a small key that reads a hostile email can still be told to misbehave. It just has less to give away.

Also keep in mind that what an agent reads goes to the model provider that runs it. Choosing what each agent gets in the first place is the part you control.

Where does Aperisa fit?

Aperisa has read-only Google access (gmail.readonly, drive.readonly, calendar.readonly, contacts.readonly) and never sends, edits or deletes anything in your Google account. It keeps a searchable copy to answer from.

Give each agent its own key and set what it can reach: which data areas, work or personal, and how much detail, from full records down to totals only. Aperisa enforces those limits on the server. Revoke an agent key and it stops working within 2 minutes.

Aperisa connects as a remote MCP server over Streamable HTTP with a Bearer key, and each key can be limited by data area and level of detail. Try it out.

FAQ

Does gmail.readonly limit what an agent can read? It limits what the agent can do, not what it can read. Google describes gmail.readonly as "View your email messages and settings" and lists it as a restricted scope, a group Google says provides "wide access to Google user data." The agent can't send or delete, but it can read any message in the mailbox.

What is least privilege for AI agents? Giving each agent only the data its job needs, and nothing it might need someday. In practice that means choosing the data areas it can reach, keeping work and personal apart, and choosing how much detail it gets, from full records down to totals.

Should every agent have its own API key? Yes, in any harness. A shared key means you can't tell which agent read what, and you can't cut off one agent without breaking the others. One key per agent lets you scope each one to its job and revoke it alone.

How do I limit a Google Drive agent to some files? Google's drive.file scope covers only files the app creates and files the user opens or shares with it through a file picker. That fits an app where a person picks files. An agent that searches across Drive needs a broader scope, so the narrowing has to happen in a layer between Google and the agent.

Sources

Questions

Does gmail.readonly limit what an agent can read?

It limits what the agent can do, not what it can read. Google describes gmail.readonly as "View your email messages and settings" and lists it as a restricted scope, a group Google says provides "wide access to Google user data." The agent can't send or delete, but it can read any message in the mailbox.

What is least privilege for AI agents?

Giving each agent only the data its job needs, and nothing it might need someday. In practice that means choosing the data areas it can reach, keeping work and personal apart, and choosing how much detail it gets, from full records down to totals.

Should every agent have its own API key?

Yes, in any harness. A shared key means you can't tell which agent read what, and you can't cut off one agent without breaking the others. One key per agent lets you scope each one to its job and revoke it alone.

How do I limit a Google Drive agent to some files?

Google's drive.file scope covers only files the app creates and files the user opens or shares with it through a file picker. That fits an app where a person picks files. An agent that searches across Drive needs a broader scope, so the narrowing has to happen in a layer between Google and the agent.

Published

Keep reading