AI Agent Risk Assessment: A Checklist Before You Allow an Agent
Someone wants an AI agent that reads their email, books travel or runs commands on a build server. Or you find one already running. You need a fast, repeatable way to say yes, no, or yes with limits.
This checklist covers eight areas, the red flags in each, and the control that matches every answer.
AI Agent Blocklist: cover every agent you have not assessed
Block AI browsers, agents, agent extensions and MCP servers by default while you review requests one by one. Each entry carries a risk grade; the lists update every day.
Why an agent needs its own risk assessment
Your vendor questionnaire was written for software that stores data. An agent does things. It acts with someone's logins, takes instructions from content it reads, and calls tools nobody listed in the contract.
What a normal SaaS review asks
- Where is the data stored and for how long?
- Which certifications does the vendor hold?
- Who at the vendor can access the data?
- Does it support single sign-on?
- What happens to the data when we leave?
What an agent review must add
- What can it do on its own, without a person approving each step?
- Whose credentials and sessions does it use?
- Which outside content can steer it: web pages, emails, files, tool output?
- Which tools and servers can it call, and who controls them?
- How do we stop it everywhere at once?
What went wrong in 2025, and the question each case adds
Each item on the checklist below comes from a failure that was widely reported. None needed an exotic attack. Each used an ability the agent was given on purpose.
- 2025
A coding agent deleted a production database during a declared code freeze, after being told not to change anything.
Adds: can it run destructive actions without a person approving them? - 2025
Malware in a compromised build package looked for AI coding command-line tools on developer machines and used them to search for secrets.
Adds: what can other software do with this agent once it is installed? - August 2025
Researchers showed that hidden text in a comment on a web page could make an AI browser read a one-time code from the user's open email and send it out.
Adds: which untrusted content can give it instructions? - September 2025
A package posing as a known email MCP server quietly copied every email sent through it to an outside address.
Adds: who publishes each tool it uses, and is the version pinned? - December 2025
Gartner told security leaders "Cybersecurity Must Block AI Browsers for Now", and OWASP published its Top 10 for Agentic Applications.
Adds: can we show an auditor what we decided and why?
For the wider picture, see our page on AI agent security risks and on Gartner's advice to block AI browsers.
The AI agent security checklist: eight areas
Work through the areas in order. Each one lists the questions to answer and the red flags that push the decision toward block. Open an area to see its questions.
1Identity and credentialsWhose access does the agent use?Questions
- Does it act through the user's own browser sessions, stored passwords or tokens?
- Can it have its own account with narrower rights instead?
- Are tokens scoped to the task, or do they grant full access to a mailbox or drive?
- Can you revoke its access without resetting the user's password?
- Uses an admin's session
- Needs a password manager or full mailbox access
- No separate identity to revoke
2Actions and autonomyWhat can it do without asking?Questions
- Can it send, buy, delete, publish, merge or run commands?
- Which of those need a person to approve each time?
- Can users switch approvals off with an "always allow" setting?
- Can an admin lock the approval settings so users cannot change them?
- Irreversible actions with no approval step
- Auto-approve available to every user
- Runs on a schedule while nobody watches
3Inputs it trustsWhat content can steer it?Questions
- Does it read open web pages, inbound email, shared files or tickets from outsiders?
- Can it act on the same page where it read that content?
- Does the vendor describe how it separates the user's request from page content?
- Can you limit the sites or sources it may act on?
- Reads any site and acts in the same session
- Processes inbound email and can also send
- No site restrictions at all
4Data flow and retentionWhere does what it sees go?Questions
- Which content leaves the device: whole pages, screenshots, files, keystrokes?
- Which countries and sub-processors handle it, and for how long?
- Is business data used to train models, and can that be switched off by contract?
- Does the agent keep memory across tasks, and can you clear it?
- Screenshots of every page sent out
- Training on business data with no opt-out
- Memory you cannot see or delete
5Tools, plugins and MCP serversWhat can it call?Questions
- Which tools ship with it, and which can users add themselves?
- Who publishes each MCP server or plugin, and is the version pinned?
- Can an admin restrict tools to an approved list?
- Do tool results get treated as instructions?
- Users add any server from a public hub
- Unpinned versions that update silently
- No admin control over the tool list
6Where it runsBrowser, desktop, terminal or cloud?Questions
- Is it a browser, an extension, a desktop app, a command-line tool or a hosted service?
- Does it run in a sandbox, or with the user's full file and shell access?
- Can its network access be limited to known destinations?
- Can your MDM install, configure and remove it?
- Full shell access on machines with production keys
- Installs per user, outside your MDM
- Unrestricted network egress
7Logging and auditCan you see what it did?Questions
- Is every action logged with time, target and the instruction behind it?
- Can the logs go to your own logging system?
- Do downstream systems show the agent, or only the user, as the actor?
- How long are logs kept, and who can read them?
- Actions look exactly like the user's own
- Logs only visible in the vendor's app
- No record of the instruction that caused an action
8Kill switch and exitHow do you stop it?Questions
- Can you turn it off centrally within minutes?
- Can you block it at the network and on devices if the vendor console fails?
- Can you revoke every token and connector it holds?
- Who owns the decision, and when is it reviewed?
- No admin console
- Off switch only in each user's settings
- No named owner on your side
MCP server security checklist
MCP servers give agents their hands: they connect to mail, files, code and cloud accounts. Assess each server as a separate piece of software, not as a feature of the agent. Our full MCP security checklist goes deeper.
Publisher
Is it from the service's own maker or a stranger with a similar name? Lookalike packages are the easiest trap.
Pinned version
Pin an exact version you reviewed. A later version can change behaviour in one line.
Token scope
Give each server the narrowest token that does the job. Never reuse an admin token.
Local or remote
Local servers run with the user's rights. Remote servers see everything you send them.
Tool descriptions
Read them. Hidden instructions in a tool's description can steer the agent.
Network destinations
List where it connects. A mail server that also talks to an unknown host is a finding.
Approval
Require a person to approve tool calls that send, delete or pay.
Logging
Log every call with its arguments, so a bad result can be traced.
Approved servers go on an allowlist; everything else stays blocked. See our MCP allowlist guide for how to enforce one.
AI browser extension risk assessment
Extensions are the quiet way agents arrive. They install in seconds, read every page, and update without asking. Rate each one by what it is allowed to do, not by its store rating.
| What the extension can do | Typical example | Risk tier | Usual decision |
|---|---|---|---|
| Read and act on any site: click, type, submit | Browser agent and "operator" extensions | Highest | Block unless formally approved |
| Read any site and send content out | AI sidebars that summarise every page | High | Block, or allow one approved tool |
| Read only when the user clicks | Tools limited to the active tab on demand | Medium | Allow per team with review |
| Work on a few named sites | Helpers for one web app | Lower | Allow if the vendor is approved |
Also check who publishes it and whether that changed recently, how often it updates, and which domains it talks to. More in are AI Chrome extensions safe.
Turn the answers into a decision
Two questions drive most outcomes: how much can the agent do without approval, and how sensitive is what it can reach? Place the agent on this grid, then apply the controls in the next section.
Any red flag in areas 1, 3 or 5 moves the agent one step toward block. Record the decision, the owner and the review date in your AI agent inventory.
Agentic AI security controls, layer by layer
"Allow with controls" means nothing until each control has a place to live. This table maps the common agentic AI security controls to the tools most admins already run.
| Layer | Control | Where it is set |
|---|---|---|
| Network | Block unassessed agent services; allow approved ones | DNS filter, firewall external lists, secure web gateway |
| Device | Install only approved AI browsers and agent apps; remove the rest | MDM and app control on Windows and macOS |
| Browser | Block agent extensions by ID; turn off built-in agent features | Chrome and Edge policies, Google Admin console, Group Policy |
| Identity | Separate, scoped accounts and tokens for agents | Your identity provider and each connected app |
| Agent config | Lock allowed tools, commands and file paths | Managed settings files, for example deny rules such as Read(./.env) in a coding agent's managed-settings.json |
| Agent egress | Limit where a cloud agent can connect | Agent firewalls, for example the GitHub Copilot cloud agent firewall under Settings, Copilot, Internet access |
| Monitoring | Log actions and watch for new agents | DNS and proxy logs, endpoint inventory, agent audit logs |
Know each control's limits. The Copilot cloud agent firewall, for example, covers processes started through the agent's Bash tool and its built-in Playwright MCP server, not other MCP servers. See also agent identity and permissions.
How we help: block by default, assess on request
The checklist works for the agents people ask about. The problem is the hundreds they never ask about. The AI Agent Blocklist covers those, so your review queue sets the pace instead of your users.
Block the unknown
Load the list into DNS, firewall, browser policy and MDM. Unassessed agents are stopped on day one.
Take requests
Staff ask for the agent they need. The request starts the eight-area review above.
Allow by exception
Approved agents go on an allow rule with an owner and a review date.
Stay current
New agents are added to the list every day, so the default stays "not yet assessed".
AI agent domains: AI browsers, autonomous agents, agent platforms, MCP servers, coding, research, meeting and sales agents.
Agent identifiers on devices: extension IDs, bundle IDs, Windows package IDs, process names and CLI packages.
On every entry, so you can block all agents or only the higher grades while reviews run.
The identifiers also help find agents already installed. Examples from the list: the OpenClaw Mac app ai.openclaw.mac and the Codex CLI Windows package OpenAI.Codex. For coding agents, see coding agent security risks.
Give your assessment process a safe default
Assess the agents people need. Block the rest automatically, with a list that grows every day.
- Domain list, hosts file, PAC file and firewall external list format
- Chrome and Edge extension block lists for Group Policy, Intune and Google Admin
- macOS and Windows identifiers for Jamf and Intune
- CSV and JSON with tool name, category, identifier type and risk grade
- Updated every day; access set up within one business day of an order
Ask us or order a plan
Tell us what you need to block and where your rules live. We reply within one business day.
- Files for firewalls, DNS filters, secure web gateways, MDM and browser policies
- Updated every day as new AI browsers, agents and extensions appear
- Plans for one organisation, for MSPs and for product makers, or a hosted filter we run for you
AI agent risk assessment: common questions
What is an AI agent risk assessment?
A short, repeatable review of one agent before it is allowed: what it can do, what it reads, where data goes, which tools it uses, where it runs, what it logs and how you switch it off. The outcome is block, allow with controls, or allow.
How is assessing an AI agent different from a normal SaaS review?
A SaaS review asks where data is stored. An agent review also asks what it does on its own with the user's logins, which outside content can steer it, and which tools it can call.
What should an MCP server security checklist include?
Publisher and source, a pinned version, token scope, local or remote running, a read of its tool descriptions, network destinations, approval for risky calls and logging of every call.
How do I assess the risk of an AI browser extension?
Check which sites it can read and change, whether it can act or only read, who publishes it and whether that changed, how it updates, and where page content goes. Extensions that click and type on any site belong in the highest tier.
Can an AI agent do the risk assessment for me?
An AI assistant can help draft answers from vendor documents, but a named person must check and sign the result. Never give an agent under review the access you are assessing it for.
What happens to agents nobody has assessed yet?
Block them by default and assess on request. A daily blocklist of AI agents, AI browsers, extensions and MCP servers covers the ones you have not reviewed.
How often should an approved agent be reassessed?
At least once a year, and whenever it gains new abilities, new tools or connectors, a new owner, or a reported security issue.
Next: write the result into your AI agent acceptable use policy, and decide whether each layer runs on an allowlist or a blocklist.
Claude Code is a trademark of Anthropic PBC. GitHub and GitHub Copilot are trademarks of GitHub, Inc. Codex is a trademark of OpenAI. Chrome is a trademark of Google LLC. Microsoft, Edge and Intune are trademarks of Microsoft Corporation.
Gartner is a trademark of Gartner, Inc. OWASP is a trademark of the OWASP Foundation. Manus, Sider, OpenClaw and Jamf are trademarks of their respective owners.
Names are used only to identify the products discussed. AI Agent Blocklist is not affiliated with, sponsored or endorsed by these companies.