The three routes, in plain terms
Pick by what you have: a developer, an enterprise budget, or neither.
1. Done-for-you audit — AgentCheck
One-time, price published. We attack your chatbot through the same public interface your customers use — 60 adversarial prompts (Mini) or 200+ across all five failure classes (Full) — and hand you a written report with prioritized fixes.
- You hand over: a link to your public bot. Nothing else
- No passwords, no code access, no calls
- Written report in 2–3 business days
- Full Audit includes a free re-check within 30 days
- Optional Agent Guard — $49/mo ongoing checks
2. Do-it-yourself — free tools
License price — your time is the cost. Promptfoo, Garak, PyRIT and DeepTeam are free, open-source red-teaming tools. A developer installs one, configures your bot as a target, runs the tests and interprets the raw output.
- You hand over: developer hours, not money
- Setup, target configuration and test design are on you
- Output is scanner data — you prioritize the fixes yourself
- Runs inside your own pipeline, including CI/CD
- Repeat runs whenever you change the bot
3. Human-led red-team firm
No published price. Firms such as Bishop Fox, Coalfire, NCC Group and Trail of Bits scope an engagement around your systems and quote it after an introductory process. Scope can go far beyond the public chat window.
- You hand over: access scoped to the engagement
- Can include source code, cloud and backend review
- Consultant-led testing and reporting
- Timeline and deliverables set during scoping
- Suited to regulated or high-risk environments
Side by side
Facts from each option's public pages and repositories, checked November 2026. Where a vendor publishes no price, the table says so rather than guessing.
| AgentCheck (service) | DIY tools (Promptfoo, Garak, PyRIT, DeepTeam) | Human-led red-team firms | |
|---|---|---|---|
| Price | $149 Mini / $299 Full, published. Guard $49/mo | Free and open source | Not published — quoted per engagement |
| What you hand over | A link to your public chatbot. No passwords, no code | Your developer's time: install, configure, run, interpret | Access scoped to the engagement; can include code and systems |
| Who runs the tests | We do, through the public chat interface | You or your developer | The firm's consultants |
| What you receive | Written report in plain English: what broke, evidence, prioritized fixes | Raw scanner output and logs to interpret yourself | Consultant report; depth set by the engagement scope |
| Turnaround | 2–3 business days | Depends on your setup time and schedule | Set during scoping; no fixed public timeline |
| Coverage | Public-interface behavior: prompt injection, data leaks, hallucinated prices and policies, jailbreaks, runaway spending | Promptfoo: application-specific attacks for apps, RAG and agents. Garak: probes for known LLM weaknesses. PyRIT: risk-identification framework for engineers. DeepTeam: OWASP/NIST-aligned framework, no UI | Can extend beyond the public interface — code, infrastructure, processes — as scoped |
| Ongoing checks | Agent Guard, $49/mo: spending caps, unusual-behavior alerts, re-tests after updates | Re-run the tools yourself after each change | Ongoing programs where offered, priced per quote |
| Fits when | You run a live, customer-facing bot and have no security staff | You have a developer who can own the tooling | You need code/cloud scope, a regulated environment, or a formal engagement |
The DIY tools, one by one
All four are free and open source. The real question is who on your team will run them.
Promptfoo
A developer-run tool that tests complete LLM systems — applications, RAG systems and agents. It supports HTTP interfaces, Python scripts and CI/CD pipelines, and generates application-specific attacks. The most flexible of the four if you have engineering support; it is a tool you operate, not a finished audit.
Garak (NVIDIA)
A free, open-source vulnerability scanner that probes whether an LLM-backed application can be made to fail — hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks. It can test REST-accessible applications, but operating it requires technical setup and a target configuration.
PyRIT (Microsoft)
The Python Risk Identification Tool: an open-source framework built for security professionals and engineers to proactively identify risks in generative-AI systems. A recognized engineering route for teams building their own testing practice — not a turnkey small-business audit.
DeepTeam
An open-source red-teaming framework aligned to the OWASP and NIST AI RMF frameworks. Public comparisons note it has no user interface and no runtime defense — it is a framework your engineers build on, with results your team interprets.
Frameworks such as the OWASP LLM Top 10 and the NIST AI Risk Management Framework are free guidance documents. They tell you what to worry about; they do not test your bot. Many teams read the frameworks, then choose one of the three routes above to do the actual testing.
What AgentCheck does not do
A public-interface audit has a boundary. Knowing it is part of choosing correctly.
Not a code or cloud audit
We test what your bot says and does through its public interface. We do not review your source code, infrastructure, or internal access controls — that is the territory of the human-led firms above.
Not a certification
The report documents what we tried, what broke, and how to fix it, with a free re-check on the Full Audit. It is not a compliance certificate and does not guarantee an agent can never fail.
Not a replacement for a full red team
If your agent handles payments, health data, or other high-risk actions at scale, a scoped human-led engagement covers ground a public-interface test cannot. For a live customer-facing bot with no testing at all, a fixed-price audit is the practical first step.
Questions buyers ask
Do you need my passwords or code?
No. AgentCheck tests your chatbot the same way a customer — or an attacker — reaches it: through the public interface. That boundary is deliberate: nothing to hand over, nothing to revoke afterwards.
The DIY tools are free. Why would anyone pay for an audit?
The license is free; the work is not. Someone has to install the tool, configure your bot as a target, design the tests, run them, and turn raw output into a fix list. If you have that developer, the DIY route is reasonable. AgentCheck is the same testing bought as a finished, written report instead.
When is a human-led firm the right choice?
When the scope has to go beyond the chat window — source code, cloud configuration, internal systems — or when a regulator, insurer, or enterprise customer expects a formal engagement from a named firm, and your budget fits a quoted project.
What exactly do I get from AgentCheck?
A written report in plain English: the tests we ran across the five failure classes, what your bot actually did, and a prioritized list of fixes. The Full Audit ($299, 200+ prompts) includes a free re-check within 30 days. Delivery is 2–3 business days.
Can I start smaller?
Yes. The free mini-check scans your bot's public pages for identification, visibly exposed sensitive content, and HTTPS/privacy basics. It is a surface scan, separate from the 60- and 200+-prompt audits. Run the free mini-check.
What happens after the report?
You fix the issues — most are prompt, configuration, and limit changes — and the Full Audit's free re-check verifies them. Agent Guard ($49/mo) adds spending caps, unusual-behavior alerts, and re-testing after updates.
Not sure which route fits?
If your bot is live and untested, start with the free mini-check — it takes a minute and shows the surface issues. If you want the full adversarial battery without running tools yourself, the fixed-price audit is ready when you are.
Sources: Promptfoo (promptfoo.dev), Garak (github.com/NVIDIA/garak), PyRIT (github.com/microsoft/PyRIT), DeepTeam (via Confident AI's public tool comparison). Human-led firms are named from their public service pages; their prices are not published. Competitor details checked November 2026 and describe each option's public positioning, not test results.