{
 "name": "Agent Evaluation Checklist",
 "edition": "2026-11-19",
 "license": "CC BY 4.0",
 "cite": "Agentic AI in Cybersecurity, Agent Evaluation Checklist, edition 19 Nov 2026, https://agenticaiincybersecurity.com/evaluation-checklist/",
 "fields": [
  "id",
  "area",
  "question",
  "why",
  "sources",
  "quotes",
  "page"
 ],
 "rows": [
  {
   "id": "Q01",
   "area": "Scope of the job",
   "question": "Does this security job need an agent at all, or would a fixed workflow with a model call inside it do the work?",
   "why": "An agent chooses its own steps, which widens what it can change. A workflow with predefined code paths keeps the set of possible actions known in advance.",
   "sources": "https://www.anthropic.com/engineering/building-effective-agents https://www.anthropic.com/engineering/building-effective-agents",
   "quotes": "Anthropic, Building effective agents: \"Workflows are systems where LLMs and tools are orchestrated through predefined code paths.\" | Anthropic, Building effective agents: \"we recommend finding the simplest solution possible, and only increasing complexity when needed.\"",
   "page": "/evaluation-checklist/need-an-agent-at-all/"
  },
  {
   "id": "Q02",
   "area": "Scope of the job",
   "question": "Which production systems and which kinds of change (read, comment, ticket, configure, block, delete) is the agent allowed to touch, written down per tool?",
   "why": "Excessive agency is traced to excessive functionality, permissions and autonomy. A per-tool list makes each of those reviewable.",
   "sources": "https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ https://genai.owasp.org/llmrisk/llm062025-excessive-agency/",
   "quotes": "OWASP LLM06:2025 Excessive Agency: \"The root cause of Excessive Agency is typically one or more of: excessive functionality; excessive permissions; excessive autonomy.\" | OWASP LLM06:2025 Excessive Agency: \"Limit the extensions that LLM agents are allowed to call to only the minimum necessary.\"",
   "page": "/evaluation-checklist/scope-of-change/"
  },
  {
   "id": "Q03",
   "area": "Scope of the job",
   "question": "Does any tool give the agent an open-ended capability (run a shell command, fetch any URL) where a narrow function would do?",
   "why": "Open-ended tools let the agent do things nobody reviewed. A narrow function limits the action space to the job.",
   "sources": "https://genai.owasp.org/llmrisk/llm062025-excessive-agency/",
   "quotes": "OWASP LLM06:2025 Excessive Agency: \"Avoid the use of open-ended extensions where possible (e.g., run a shell command, fetch a URL, etc.) and use extensions with more granular functionality.\"",
   "page": "/evaluation-checklist/scope-of-change/"
  },
  {
   "id": "Q04",
   "area": "Scope of the job",
   "question": "What identity and permissions does the agent use on each downstream system, are they the least needed (for example read-only on a log store, write on one ticket queue), and is authorization enforced by those systems rather than left to the model?",
   "why": "The agent can do whatever its credentials allow, whatever its instructions say.",
   "sources": "https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml https://saif.google/focus-on-agents https://genai.owasp.org/llmrisk/llm062025-excessive-agency/",
   "quotes": "OWASP LLM06:2025 Excessive Agency: \"Limit the permissions that LLM extensions are granted to other systems to the minimum necessary in order to limit the scope of undesirable actions.\" | MITRE ATLAS mitigations: \"the principle of least privilege so that the agent is only granted the necessary permissions to access tools and resources required to accomplish its designated task(s).\" | Google, Secure AI Framework: \"Tools are the external APIs and services an agent uses to take action in the world, which must be secured with least-privilege permissions.\" | OWASP LLM06:2025 Excessive Agency: \"Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not.\"",
   "page": "/evaluation-checklist/tool-permissions-and-identity/"
  },
  {
   "id": "Q05",
   "area": "Approval and handover",
   "question": "Which actions need a named human to approve them before they run, and which may run unattended?",
   "why": "Guidance asks for approval of high-impact actions before they are taken, and for the approval rule to follow consequence.",
   "sources": "https://genai.owasp.org/llmrisk/llm062025-excessive-agency/ https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml",
   "quotes": "OWASP LLM06:2025 Excessive Agency: \"Utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken.\" | MITRE ATLAS mitigations: \"Systems should require the user or another human stakeholder to approve AI agent actions before the agent takes them.\" | MITRE ATLAS mitigations: \"Human In-the-Loop policies should follow the degree of consequence of the task at hand.\"",
   "page": "/evaluation-checklist/human-approval-points/"
  },
  {
   "id": "Q06",
   "area": "Approval and handover",
   "question": "Is each tool rated for risk (read or write, reversible or not, which accounts it needs), and do those ratings drive when the agent must pause?",
   "why": "A rating turns the approval policy into something the harness can enforce rather than a document.",
   "sources": "https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf",
   "quotes": "OpenAI, A practical guide to building agents: \"based on factors like read-only vs. write access, reversibility, required account permissions, and financial impact.\" | OpenAI, A practical guide to building agents: \"Actions that are sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent’s reliability grows.\"",
   "page": "/evaluation-checklist/human-approval-points/"
  },
  {
   "id": "Q07",
   "area": "Approval and handover",
   "question": "What makes the agent stop and hand over (retry limits, rate limits, unexpected tool results), and who is paged when it does?",
   "why": "Limits on retries and on action rate cap the damage per unit time and give monitoring time to notice.",
   "sources": "https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf https://genai.owasp.org/llmrisk/llm062025-excessive-agency/",
   "quotes": "OpenAI, A practical guide to building agents: \"Exceeding failure thresholds: Set limits on agent retries or actions.\" | OWASP LLM06:2025 Excessive Agency: \"Implement rate-limiting to reduce the number of undesirable actions that can take place within a given time period\"",
   "page": "/evaluation-checklist/stop-and-handover-conditions/"
  },
  {
   "id": "Q08",
   "area": "Approval and handover",
   "question": "Can the approver understand what the agent is proposing and why, and are they trained to resist over-trusting its output?",
   "why": "Approval is only a control if the person can interpret the proposal, disagree and say no. Note: Article 14 applies to systems the Act classifies as high risk; for others it is a reference point only.",
   "sources": "https://artificialintelligenceact.eu/article/14/ https://artificialintelligenceact.eu/article/14/ https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf",
   "quotes": "EU AI Act, Article 14: \"to remain aware of the possible tendency of automatically relying or over-relying on the output\" | EU AI Act, Article 14: \"to decide, in any particular situation, not to use the high-risk AI system\" | NIST AI RMF 1.0: \"GOVERN 3.2: Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems.\"",
   "page": "/evaluation-checklist/human-approval-points/"
  },
  {
   "id": "Q09",
   "area": "Evidence and monitoring",
   "question": "Is every step the agent takes (inputs, intermediate decisions, tool calls with parameters, data touched, identity used) logged somewhere the agent cannot edit?",
   "why": "Without a step-level record you cannot reconstruct what changed or review a disputed change.",
   "sources": "https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml https://www.anthropic.com/engineering/building-effective-agents",
   "quotes": "MITRE ATLAS mitigations: \"When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent.\" | Anthropic, Building effective agents: \"Prioritize transparency by explicitly showing the agent’s planning steps.\"",
   "page": "/evaluation-checklist/evidence-and-audit-trail/"
  },
  {
   "id": "Q10",
   "area": "Evidence and monitoring",
   "question": "After each change, what independent check (a query against the real system, a test, a diff) confirms the change happened as claimed, rather than trusting the agent’s own report?",
   "why": "An agent can state success without having achieved it. The outcome is the state of the environment, not the transcript.",
   "sources": "https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents https://www.anthropic.com/engineering/building-effective-agents",
   "quotes": "Anthropic, Demystifying evals for AI agents: \"A flight-booking agent might say “Your flight has been booked” at the end of the transcript, but the outcome is whether a reservation exists in the environment’s SQL database.\" | Anthropic, Building effective agents: \"it's crucial for the agents to gain “ground truth” from the environment at each step\"",
   "page": "/evaluation-checklist/verify-after-each-change/"
  },
  {
   "id": "Q11",
   "area": "Evidence and monitoring",
   "question": "What do you measure after go-live to notice that the agent’s behavior or quality has shifted, and who reviews it on what schedule?",
   "why": "Behavior can change with data, model or prompt changes; the guidance asks for monitoring plans in operation.",
   "sources": "https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-operation-maintenance https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf",
   "quotes": "NCSC, Guidelines for secure AI system development: \"You measure the outputs and performance of your model and system such that you can observe sudden and gradual changes in behaviour affecting security.\" | NIST AI RMF 1.0: \"MANAGE 4.1: Post-deployment AI system monitoring plans are implemented\" | NIST AI 600-1: \"MG-4.1-002 Establish, maintain, and evaluate effectiveness of organizational processes and procedures for post-deployment monitoring of GAI systems\"",
   "page": "/guides/measuring-an-agent-after-it-ships/"
  },
  {
   "id": "Q12",
   "area": "Stop and roll back",
   "question": "Can one person stop the agent immediately, and can every class of change it makes be undone or compensated for?",
   "why": "Stopping the agent is not enough if its earlier changes persist. Reversibility is also a factor in how much autonomy a tool should get.",
   "sources": "https://artificialintelligenceact.eu/article/14/ https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf",
   "quotes": "EU AI Act, Article 14: \"or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.\" | NIST AI RMF 1.0: \"to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use.\"",
   "page": "/evaluation-checklist/kill-switch-and-rollback/"
  },
  {
   "id": "Q13",
   "area": "Stop and roll back",
   "question": "Is there an incident plan for the case where the agent itself caused the incident, with criteria for switching it off and a path for remediation?",
   "why": "Guidance expects incident, escalation and remediation plans that cover the AI system, with set criteria for deactivation.",
   "sources": "https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-deployment https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf",
   "quotes": "NCSC, Guidelines for secure AI system development: \"The inevitability of security incidents affecting your AI systems is reflected in your incident response, escalation and remediation plans.\" | NIST AI 600-1: \"MG-2.4-002 Establish and maintain procedures for escalating GAI system incidents to the organizational risk management authority when specific criteria for deactivation or disengagement is met\" | NIST AI RMF 1.0: \"GOVERN 6.2: Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk.\"",
   "page": "/evaluation-checklist/kill-switch-and-rollback/"
  },
  {
   "id": "Q14",
   "area": "Evaluation",
   "question": "What written success criteria and test set must the agent meet for this specific job before it gets production access, and were they measured in conditions like your environment?",
   "why": "Generic benchmark results do not show how the agent handles your alerts, tickets and systems.",
   "sources": "https://platform.claude.com/docs/en/test-and-evaluate/develop-tests https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf",
   "quotes": "Anthropic, Define success criteria and build evaluations: \"Building a successful LLM-based application starts with clearly defining your success criteria and then designing evaluations to measure performance against them.\" | NIST AI RMF 1.0: \"MEASURE 2.3: AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s).\" | NIST AI 600-1: \"Avoid extrapolating GAI system performance or capabilities from narrow, non-systematic, and anecdotal assessments.\"",
   "page": "/evaluation-checklist/success-criteria-and-test-sets/"
  },
  {
   "id": "Q15",
   "area": "Evaluation",
   "question": "Has the agent been run against copies or sandboxes of the systems it will change, with its write tools pointed at the copy, before any real change?",
   "why": "Errors compound across steps; the vendor guidance recommends sandboxed testing alongside guardrails.",
   "sources": "https://www.anthropic.com/engineering/building-effective-agents https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-deployment",
   "quotes": "Anthropic, Building effective agents: \"The autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails.\" | MITRE ATLAS mitigations: \"Define security boundaries around agentic tools and data sources with methods such as API access, container isolation, code execution sandboxing, and rate limiting of tool invocation.\" | NCSC, Guidelines for secure AI system development: \"You release models, applications or systems only after subjecting them to appropriate and effective security evaluation such as benchmarking and red teaming\"",
   "page": "/evaluation-checklist/success-criteria-and-test-sets/"
  },
  {
   "id": "Q16",
   "area": "Evaluation",
   "question": "What re-runs the evaluation when the model version, prompt, tools or data change, and what result blocks the change from shipping?",
   "why": "Changes to data, models or prompts can change behavior, so each needs the same gate as a new version.",
   "sources": "https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-operation-maintenance https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-218A.pdf https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents",
   "quotes": "NCSC, Guidelines for secure AI system development: \"you treat major updates like new versions\" | NIST SP 800-218A: \"C1: Consider automating tests within a development pipeline as part of regression testing where possible.\" | Anthropic, Demystifying evals for AI agents: \"Once evals exist, you get baselines and regression tests for free\"",
   "page": "/evaluation-checklist/re-evaluate-after-model-changes/"
  },
  {
   "id": "Q17",
   "area": "Data and inputs",
   "question": "What data can the agent read for this job, does it include production or sensitive records, and who approved that access?",
   "why": "Data the agent reads can leak through its outputs and its tool calls; guidance asks for controls on sensitive data used in AI work.",
   "sources": "https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-218A.pdf https://saif.google/focus-on-agents https://www.ncsc.govt.nz/assets/guidance/Documents/csi-deploying-ai-systems-securely.pdf",
   "quotes": "NIST SP 800-218A: \"R2: Only store sensitive data used during AI model development, including production data, within organization-approved environments and locations within those environments.\" | Google, Secure AI Framework: \"including enforcing permissions on the agent’s access to tools and sensitive data\" | CISA, Deploying AI Systems Securely: \"Secure sensitive AI information (e.g., AI model weights, outputs, and logs) by encrypting the data at rest, and store encryption keys in a hardware security module (HSM) for later on-demand decryption\"",
   "page": "/evaluation-checklist/tool-permissions-and-identity/"
  },
  {
   "id": "Q18",
   "area": "Data and inputs",
   "question": "Which text the agent ingests (alerts, tickets, emails, logs, web pages) can come from outside your control, and what do its tools do when such text is in context?",
   "why": "Content from outside can steer an agent that has write tools. The mitigations restrict what tools may run in that state.",
   "sources": "https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml https://genai.owasp.org/llmrisk/llm062025-excessive-agency/",
   "quotes": "MITRE ATLAS mitigations: \"It is recommended that tool invocation be restricted or limited when untrusted data enters the LLM's context.\" | OWASP LLM06:2025 Excessive Agency: \"direct/indirect prompt injection from a malicious user, an earlier invocation of a malicious/compromised extension\"",
   "page": "/evaluation-checklist/untrusted-input/"
  },
  {
   "id": "Q19",
   "area": "Build or buy",
   "question": "If the agent is bought or hosted by a supplier, what do you control (tools, permissions, approval gates, logs, update timing) and what does the supplier control?",
   "why": "Risk is spread across third-party software, data and models, so the split needs to be written down.",
   "sources": "https://www.ncsc.govt.nz/assets/guidance/Documents/csi-deploying-ai-systems-securely.pdf https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-operation-maintenance",
   "quotes": "CISA, Deploying AI Systems Securely: \"Require the primary developer of the AI system to provide a threat model for their system.\" | NIST AI RMF 1.0: \"MAP 4: Risks and benefits are mapped for all components of the AI system including third-party software and data.\" | NCSC, Guidelines for secure AI system development: \"You support users to evaluate and respond to model changes (for example by providing preview access and versioned APIs).\"",
   "page": "/evaluation-checklist/supplier-controls-when-buying/"
  },
  {
   "id": "Q20",
   "area": "Build or buy",
   "question": "If you build, which parts (model, harness, tools, sandbox) can be swapped later, and what has the supplier told you about known limitations and failure modes?",
   "why": "Separating the parts makes upgrades and exits cheaper, and stated limitations tell you what to test.",
   "sources": "https://www.anthropic.com/engineering/managed-agents https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-deployment https://www.anthropic.com/engineering/building-effective-agents",
   "quotes": "Anthropic, Scaling Managed Agents: \"This allows the implementation of each to be swapped without disturbing the others.\" | NCSC, Guidelines for secure AI system development: \"you are clear to your users about known limitations or potential failure modes.\" | Anthropic, Building effective agents: \"Frameworks can help you get started quickly, but don't hesitate to reduce abstraction layers and build with basic components as you move to production.\"",
   "page": "/guides/build-vs-buy-ai-agents/"
  }
 ]
}