# Designing the Approval Step for a Security Agent

An approval step controls an agent only if the approver can refuse on evidence. Show the diff, the reason, the checks already run, the undo path and what the agent could not determine. Then sample accepted items: reviewing 59 finds a 5% error rate with 95% probability, reviewing 20 finds it 64% of the time. Read 19 Nov 2026.

## What the approver needs on one screen

MITRE ATLAS mitigation AML.M0029 says approval should "follow the degree of consequence of the task at hand", and that final adjudication "should be conducted by a human decision-maker". A person can only adjudicate what they can see. Five items belong on the screen where the click happens, not behind a link:

- The exact change: the diff for code, the rule text for a firewall, the setting before and after for cloud configuration.
- The reason in plain words, tied to the data the agent read: the finding, the asset, the exposure condition.
- The checks already run and their result: build, tests, rescan.
- The undo path and how long it takes.
- What the agent could not determine. An approver who is told nothing is uncertain will assume nothing is.

The agent's own account of its work is a claim. Anthropic's evals post makes the point with a flight-booking agent: it may say the flight is booked, "but the outcome is whether a reservation exists" in the database. Fill the checks line from the system of record (the rescan, the deployed version), not from the agent's summary.

## A filled approval record

The example is the Next.js path traversal CVE-2026-75604 (CVE record published 1 Sep 2026; fixed versions 15.5.24 and 16.3.3, released 25 Aug 2026). Facts about the CVE come from the Next.js advisory GHSA-p293-qw3h-jr36 as recorded in this site's CVE data, read 19 Nov 2026. The service name, installed version and times are invented for the example.

APPROVAL RECORD AP-0412
Proposed by: remediation agent, run 7f3a, 19 Nov 2026 09:14
Finding: CVE-2026-75604, Next.js incremental-cache path traversal, CVSS 9.0 (GitHub CNA)
Asset: orders-web, Windows host, internet-facing
Change: package.json next 15.5.9 -> 15.5.24; lockfile regenerated; rebuild and redeploy
Why: affected range is >= 13.4.0 and < 15.5.24 on a Windows filesystem. The vendor states no workaround exists for affected Windows-hosted applications.
Checks already run: build passed; 212 of 212 tests passed; rescan of the staging build shows next 15.5.24
Not determined by the agent: whether the app uses the App Router with Cache Components (the advisory covers Pages Router, or App Router without Cache Components); the router setting is not in the files the agent read
Undo: redeploy build 0411; about 10 minutes
Decision options: approve / edit / reject with reason
Approver: ____  Time seen: ____  Decision: ____  Reason if edited or rejected: ____

The line that earns its place is "Not determined by the agent". It tells the approver exactly what to check before clicking: the router setting decides whether the host is in scope at all. Rejecting with a reason returns feedback to the agent; editing narrows the change instead of discarding it. LangChain's human-in-the-loop docs add a caution for edits: "make changes conservatively", because large changes to the arguments can make the model re-evaluate and run the tool more than once.

## Where vendors place the approval

Each row is the vendor's own description of its product, read 19 Nov 2026, listed alphabetically.

Vendor descriptions of their own products from the pages in the source list, read 19 Nov 2026. No behavior was tested.

| Vendor | Placement as described | What the approver judges |
|---|---|---|
| Aikido | Each allowed action is set to Fully Automated or Human Approval; the fix preview shows the full diff and a confidence figure, and lower confidence is flagged for a closer look | One action at a time, with a diff |
| Astelia | The drafted ticket and the proposed firewall rule wait in an approval queue for a human click, with an execution log and an audit log | A ticket and a rule, away from the agent's context |
| Snyk | The remediation agent opens pull requests and developers review and merge | A code diff, checked by builds and tests |
| Wiz | High-confidence remediation can run automatically; lower-confidence decisions are routed to a developer or a human approval step with context and a recommendation | A recommendation with its confidence |

## How many accepted items to sample

A queue invites clicking. The check is a second look at items that were already approved. How many is a probability question. If a fraction p of accepted changes is wrong, a random sample of n finds at least one with probability 1 minus (1 minus p) to the power n. The table is that formula worked out.

Chance a random sample contains at least one wrong item, from 1 - (1 - p)^n. Arithmetic by this site, not a published figure.

| True error rate in accepted items | 20 items reviewed | 59 items reviewed | 100 items reviewed |
|---|---|---|---|
| 2% | 33.2% | 69.6% | 86.7% |
| 5% | 64.2% | 95.2% | 99.4% |
| 10% | 87.8% | 99.8% | 100% |

Rule of thumb from the table: 59 items per review period detect a 5% error rate with 95% probability, and 20 items miss it more than a third of the time. If fewer than 59 changes are accepted in a period, re-read all of them. Record the result as the revert rate in [measuring an agent after it ships](/guides/measuring-an-agent-after-it-ships/). A review that finds a wrong item is a reason to read the approver's decisions for that day, not only to fix the item.

## What happens when nobody answers

Decide in advance what an unanswered request becomes. Letting a request expire to "approved" turns the control into a timer; expire to "rejected" or escalate to a named second approver after a stated time. Log who approved, when, what they saw and what they changed in a store the agent cannot edit. MITRE ATLAS mitigation AML.M0024 asks for logging of "the intermediate steps of agentic actions and decisions, data access and tool use" and the "identity of the agent"; the [run log guide](/guides/agent-run-logs-what-to-keep/) lists the fields.

Two failures show up in the log before they show up in incidents. A refusal rate of zero for months suggests clicking, not judging. A decision time that falls while the revert rate rises means approvers are reading the summary. When an approval is raised inside a nested agent, the OpenAI Agents SDK surfaces it on the outer run, so check that the approver sees the nested action. Which actions need a click at all is set on the [human approval points](/evaluation-checklist/human-approval-points/) page.

## Sources

- [MITRE ATLAS data release 5.6.0, mitigations AML.M0029 and AML.M0024](https://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yaml), read 19 Nov 2026
- [Anthropic, Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents), read 19 Nov 2026
- [LangChain docs, Human-in-the-loop, interrupt decision types](https://docs.langchain.com/oss/python/langchain/human-in-the-loop), read 19 Nov 2026
- [OpenAI Agents SDK, Human-in-the-loop (nested agents)](https://openai.github.io/openai-agents-python/human_in_the_loop/), read 19 Nov 2026
- [Next.js security advisory GHSA-p293-qw3h-jr36 (CVE-2026-75604)](https://github.com/vercel/next.js/security/advisories/GHSA-p293-qw3h-jr36), read 19 Nov 2026
- [Aikido, AutoFix](https://www.aikido.dev/code/autofix), read 19 Nov 2026
- [Aikido, CVE Exploitability Analysis](https://help.aikido.dev/aikido-agent/cve-exploitability-analysis.md), read 19 Nov 2026
- [Snyk, Evo Remediation Agent open preview](https://snyk.io/blog/evo-remediation-agent-open-preview/), read 19 Nov 2026
- [Astelia, Closing the exposure loop](https://www.astelia.io/post/closing-the-exposure-loop), read 19 Nov 2026
- [Wiz, Introducing Wiz agents](https://www.wiz.io/blog/introducing-wiz-agents), read 19 Nov 2026

Last reviewed 19 Nov 2026

Canonical: https://agenticaiincybersecurity.com/guides/human-approval-design/
