How to use it
Ask your chatbot the questions customers actually ask — “How much is…?”, “Can I get a refund?”, “Are you open on Sunday?” — and copy its answers word for word. In the other box, paste the source of truth: the same price or policy exactly as it appears on your website, price list or policy page. Start with money topics: prices, discounts and refunds are the answers a business can be held to. In 2024 a tribunal ordered Air Canada to honor a refund its chatbot had incorrectly promised a customer — the bot's words were treated as the company's words.
One rule runs the whole score: a number in the bot's answer that is not in your source text — a different price, a discount you never approved, a longer refund window — is a Mismatch, however friendly the rest of the answer sounds. Friendly and wrong is still wrong.
Not sure what your setup allows the bot to do in the first place? Run the AI Agent Permissions Checker as well — it scores connections, keys, limits and logging. The Self-Test Scorecard covers pushy and tricky prompts.
The example is invented on purpose, so you can see a mismatch flagged before you paste your own answers.
Your score
– / 100
Fix list, in priority order
Want the bot's answers checked for you, in writing?
This checker compares the answers you paste. The AgentCheck Audit ($149 one-time) goes further: we talk to your bot through its public chat window the way customers do — ordinary questions and the awkward, pushy ones — compare its answers against your website, and deliver a written report with the actual answers as evidence and a step-by-step fix plan. The AgentCheck Full Audit ($299 one-time) adds the setup review and a free re-check within 30 days. Agent Guard ($49/month, cancel anytime) re-checks after updates. No passwords, no API keys and no code access are ever requested; full refund if work hasn't started yet.
This tool is general information about reviewing a chatbot's answers. It is not legal or security advice and not a certification. It compares the two texts you paste; it cannot tell which one is right, and a matching score does not guarantee the bot answers every question correctly. AgentCheck is not affiliated with, endorsed by, or certified by OWASP or NIST; its checklists are built on those public frameworks.
Frequently asked questions
What does the Chatbot Answer Match Checker do?
You paste, side by side, what your business actually says about a topic (a price, your refund policy, your opening hours) and what your chatbot answered a visitor about the same topic. The checker compares the two texts in your browser: numbers that appear in the bot's answer but not in your source text, source numbers the bot left out, promise words the bot added, and wording that has almost nothing in common. Each topic gets a verdict — Match, Check wording or Mismatch — plus an overall score and a fix list in priority order. Nothing is uploaded.
Does this tool talk to my chatbot?
No. It never connects to your chatbot, your website or any account. You paste both texts yourself, which also means you choose exactly which answers to check — start with prices, discounts and refunds, the answers a business can be held to.
What counts as a mismatch?
The clearest mismatch is a number: the bot quotes $35, 20% off or 30 days where your own source text says something else or says nothing at all. A flipped no is just as serious — your policy says no refunds and the bot offers one. Softer findings, like a source number the bot skipped or wording that barely overlaps, are marked Check wording rather than Mismatch.
Can the checker tell which text is right?
No. It treats the text you paste as your source of truth as correct by definition, and flags where the bot's answer departs from it. If your source text itself is out of date, fix that first — then re-check the bot against the corrected version.
How is this different from the Permissions Checker and the Self-Test Scorecard?
The Permissions Checker scores what your agent is connected to and allowed to do. The Self-Test Scorecard records how your bot handles pushy and tricky prompts. This checker covers the third AgentCheck check: whether the bot's everyday answers about prices, policies and hours match what your business actually says.
Is a matching score a guarantee that my bot answers correctly?
No. This is a self-check on the answers you pasted, on one day. The bot may answer the same question differently tomorrow, or answer a topic you did not paste wrongly. It is not a certification and not legal or security advice. The AgentCheck Audit ($149 one-time) tests the bot from the visitor's side across all five checks and delivers a written report.