AgentCheck · Free tool

AI agent permissions checker: score what your chatbot can reach

Twenty plain-English questions about your agent's connections, the actions it can take alone, its API keys, its limits and its logs. You get a 0–100 score and a fix list in priority order. Everything runs in your browser — no answers are uploaded, stored or sent anywhere.

Start the checkerRead the checklist first

How to use it

Open your chatbot or agent platform's integrations and tool settings in another tab, then answer for the setup as it is today — not as you intend it to be. “Don't know” is an honest answer and the checker treats it as a finding, because a permission nobody can describe is a permission nobody is controlling. Prefer the full explanation of each check? The Permissions Audit Checklist walks through the same ground step by step.

One rule runs the whole score: if your agent can move money, delete data, change access, export customers, speak for the business outside the chat, or change its own rules — without a human approving that specific action — the result is Critical, however tidy everything else looks.

Part 1 — The connection map (7 questions)

For each connection, choose the strongest thing the agent can do with it today.

Connections

Customer data / CRM — records, contacts, order history

Email / inbox — sending as the business

Calendar / booking — appointments and reservations

Files / documents — folders, drives, shared storage

Orders / payments — refunds, charges, discounts

Accounts / admin — users, roles, passwords

Web / other tools & APIs — sites and services it can call

Part 2 — The never-alone list (6 questions)

Can the agent do each of these without a human approving that specific action?

Never-alone actions

Move money — refunds, payments, payouts, large discounts

Delete or overwrite data — records, files, listings, backups

Change access — invite users, change roles, reset credentials, widen its own access

Export customer data — download or send a list or history anywhere

Speak for the business outside the chat — post publicly, reply to reviews, email non-participants

Change its own rules — edit its instructions, limits, allowlists or tools

Part 3 — Keys, limits and logs (7 questions)

Keys

Are any API keys stored in the agent's prompt, instructions or knowledge files?

Is each key scoped to one service and the smallest job it needs?

Could you rotate (replace) every key within minutes if one leaked tonight?

Limits and logging

Is there a spending cap set in the platform or provider settings (not just asked for in the prompt)?

Are there step or time caps, so a looping task ends by itself?

Are the agent's actions logged somewhere the agent itself cannot edit?

Would a human be alerted on a refund attempt, an export, or repeated failed actions?

Answered 0 of 20. You can score a partial run — unanswered questions count as “don't know” and are listed as findings.

Your score

– / 100  

Fix list, in priority order

Checking what the bot is allowed to do is only half of it. Also compare what it actually says: the free Chatbot Answer Match Checker puts your real prices and policies next to the bot’s answers and flags invented numbers and added promises.

Want this setup review done for you, in writing?

This checker scores your own answers. The AgentCheck Full Audit ($299 one-time) does the review for you: based on the details you share, we map every connection, note where the keys live, check the hard limits and the logging, and deliver a written report with a prioritized fix plan and a free re-check within 30 days. The AgentCheck Audit ($149 one-time) reviews the bot from the visitor's side through its public chat window. Agent Guard ($49/month, cancel anytime) re-checks after updates. No passwords, no API keys and no code access are ever requested; full refund if work hasn't started yet.

Start the Full Audit — $299Run the free mini-check first

This tool is general information about reviewing an AI agent's setup. It is not legal or security advice and not a certification. A score is a snapshot of your answers about one day's setup; a clean result does not guarantee an agent is safe. AgentCheck is not affiliated with, endorsed by, or certified by OWASP or NIST; its checklists are built on those public frameworks.

Frequently asked questions

What does this checker do?

It asks 20 questions about what your agent is connected to and allowed to do, scores the setup 0–100, labels it Passed, Warning or Critical, and orders the fixes. Nothing is uploaded.

What makes a setup Critical?

Any never-alone action the agent can do on its own — money, deletion, access changes, exports, publishing in your name, or rewriting its own rules — or an API key living in the prompt.

Is a good score a guarantee?

No. It is a self-check on your own answers about one day's setup, not a certification and not security advice. Changes to instructions, model or tools can open new holes later.

How is this different from the checklist?

The checklist page explains each check in depth. This checker turns the same checks into questions and scores them. The Full Audit ($299) does the whole review for you in writing.