Agent evaluation checklist for security teams
The checklist has 20 questions in 7 areas that a security team settles in writing before an AI agent may change a production system. Each question has a test to run on your own systems and a line quoted from NIST, OWASP, MITRE, NCSC, the EU AI Act, CISA, Anthropic or OpenAI guidance, read 19 Nov 2026.
Download the Agent Evaluation Checklist: CSV JSON PDF .docx CC BY 4.0
How to answer it
Answer each question once per security job, not once per product. An agent that triages findings and one that edits cloud configuration have different tools, approvers and rollback paths. A question is settled when the answer names a system, a setting or a person: "approvals are required" is not an answer, while "the change queue requires the service owner to approve any write tool, and the harness blocks the call until it does" is. For each question record the answer, a link to the setting or log that proves it, the owner and the date. Write "not applicable" with a reason rather than leaving a blank.
The scope is a team deploying an agent for one job; the use-case list names the six jobs published agents are described for. The checklist does not judge products. The test under each question is run on your own systems, in a test tenant where it writes anything.
What the checklist does not settle
Passing all 20 does not show the agent is good at the job: that comes from the test set built under Q14, using the guide to evaluating an agent on your findings. For the money side of buying, see the build-versus-buy guide.
In this section
- Does the Security Job Need an Agent? Checklist Question Q01
Q01: split one real finding, CVE-2021-44228, into fixed and variable steps and check whether a workflow with one model call does the job before an agent.
- Scope of Change: What an AI Agent May Touch, Per Tool
Checklist Q02 and Q03: give each tool one system and one of six change kinds (read to delete), then replace a shell tool using 412 logged calls over 30 days.
- Human Approval Points: Checklist Questions Q05, Q06, Q08
Where an agent must pause for a person: the approval map to ask for, the risk rating that drives it, and a test that shows whether approvers decide or click.
- Stop and Handover Conditions for a Security Agent
Checklist Q07: the retry, iteration and rate limits that halt an agent, a filled-in handover note, and a stop on CVE-2026-123158 timed from 09:21 to 09:38.
- AI Agent Audit Trail: A Log Entry You Can Reconstruct
Checklist question Q09 with a complete example log entry for one agent change, five reconstruction questions and a retention period worked to a date.
- Verify After Each Change: Check the System, Not the Agent
Checklist Q10: the separate check that confirms an agent's change happened, worked through on a Next.js upgrade for CVE-2026-75604, and how each check can fail.
- Kill Switch and Rollback: Checklist Questions Q12 and Q13
What to ask for before an agent changes production: who can stop it, how each change is undone, and an incident plan with switch-off criteria and a drill.
- Success Criteria and Test Sets for a Security Agent
Checklist Q14 and Q15: a criteria template, a 40-case test set run 3 times (109 of 120 trials passed) and why that still failed, with sandbox checks.
- Re-Evaluate After Model Changes: Questions Q16 and Q11
Which changes re-run the test set, what result blocks release, and how to catch a supplier-side change you never scheduled, with a worked run of 48 cases.
- Supplier Controls: Questions to Ask AI Agent Vendors
Checklist Q19 and Q20: nine questions for an AI agent supplier, each with an acceptance test and contract wording, and a one-day re-test window worked out.
- Tool Permissions and Identity: What an Agent Can Do
Checklist Q04: which identity and permissions a security agent uses on each system, with a worked AWS read-only policy, a refused-action log line, review dates.
- Untrusted Input: What Outside Text Can Make an Agent Do
Checklist Q18: which text a security agent reads comes from outside your control and which tools stay active then, with a harness rule and an 18-alert replay.
Sources
- Anthropic, Building effective agents, read 19 Nov 2026
- OWASP, LLM06:2025 Excessive Agency, read 19 Nov 2026
- OpenAI, A practical guide to building agents, read 19 Nov 2026
- MITRE ATLAS mitigations, ATLAS data release 5.6.0 (dist/ATLAS.yaml; the current release, v2026.10, renumbers some mitigations, so IDs and wording here follow 5.6.0), read 19 Nov 2026
- EU AI Act, Article 14 Human oversight (artificialintelligenceact.eu), read 19 Nov 2026
- NIST AI RMF 1.0 (NIST AI 100-1), read 19 Nov 2026
- NIST AI 600-1, Generative AI Profile, read 19 Nov 2026
- NIST SP 800-218A, read 19 Nov 2026
- NCSC, Guidelines for secure AI system development: secure operation and maintenance, read 19 Nov 2026
- NCSC, Guidelines for secure AI system development: secure deployment, read 19 Nov 2026
- Anthropic, Demystifying evals for AI agents, read 19 Nov 2026
- Anthropic, Define success criteria and build evaluations, read 19 Nov 2026
- Anthropic, Scaling Managed Agents, read 19 Nov 2026
- Google, Secure AI Framework, Focus on Agents, read 19 Nov 2026
- NSA AISC, CISA and partners, Deploying AI Systems Securely (April 2024; copy hosted by NCSC NZ), read 19 Nov 2026
Last reviewed 19 Nov 2026