# Measuring a Security Agent After It Ships

After go-live, track six numbers: acceptance rate, revert rate, human minutes per accepted change, escalation rate, safeguard block rate and cost per task. Set each threshold from your own baseline before launch. Rival's agents normally saw about 5% of threads blocked by a model safeguard, and 19.2% on 5 Nov 2026. Read 19 Nov 2026.

## Why measure outputs, not only uptime

The UK NCSC's secure AI guidelines say to measure "the outputs and performance of your model and system such that you can observe sudden and gradual changes in behaviour affecting security", and to treat "major updates like new versions" because changes to data, models or prompts change behavior. NIST AI RMF 1.0 control MANAGE 4.1 asks for "post-deployment AI system monitoring plans". Neither names a metric. The six below are this site's, chosen because each moves when a specific thing breaks.

## The six metrics, thresholds and actions

Count a change as accepted only after the outcome is checked in the system, as in [verify after each change](/evaluation-checklist/verify-after-each-change/). Thresholds are starting values chosen by this site, not published limits. Replace them with a band taken from the held-back set in the [evals guide](/guides/ai-agent-evals/) and your first four weeks.

Formulas and thresholds are this site's. The source for the reason to measure outputs is the NCSC guideline, read 19 Nov 2026.

| Metric | Formula | Starting threshold | Action when crossed |
|---|---|---|---|
| Acceptance rate | accepted proposals / proposals reviewed | Down 10 points from baseline for 2 weeks | Rerun the held-back eval set; check whether inputs or the model changed |
| Revert rate | accepted changes later undone / accepted changes | Above 5% in any week | Review a sample of 59 accepted changes; stop widening what runs unattended |
| Human minutes per accepted change | review minutes / accepted changes | Down 30% while revert rate is up | Reviewers are reading summaries: require the evidence panel to be opened |
| Escalation rate | runs that hit a stop condition / runs | Double the baseline | Inspect the stop reasons; scope is too tight or inputs changed |
| Safeguard block rate | threads refused by the model provider / threads | Double the baseline | Retry, then switch to the fallback route; log which task types were blocked |
| Cost per task | model spend / tasks completed | 1.5 times baseline | Read the [cost guide](/guides/cost-of-running-security-agents/); check retries and context growth |

Each metric needs a baseline and an owner before go-live. The baseline is the median of the weekly values from the first four weeks after go-live in which the model, prompt and tools did not change, using only weeks with at least 100 reviewed proposals (or, for the block rate, at least 200 threads); a thinner week is pooled with the next one. Until four such weeks exist, use the held-back eval set result and mark the metric provisional.

Owners, windows and minimum counts are this site's starting values, to be replaced with your own.

| Metric | Owner | Baseline window and minimum count |
|---|---|---|
| Acceptance rate | Job owner | Median of 4 clean weeks, at least 100 proposals reviewed per week |
| Revert rate | Owner of the system the change lands in | Median of 4 clean weeks, at least 100 accepted changes per week; a week with fewer is pooled |
| Human minutes per accepted change | Lead reviewer | Median of 4 clean weeks, at least 100 accepted changes per week |
| Escalation rate | Engineer who owns the harness | Median of 4 clean weeks, at least 100 runs per week |
| Safeguard block rate | Engineer who owns the harness | Median daily rate over 4 clean weeks, at least 200 threads per day; low-volume days left out, as Rival does |
| Cost per task | Engineer who owns the harness, with the budget holder | Median of 4 clean weeks, at least 100 completed tasks per week |

## A four-week review worked through

The inputs below are invented for the example; the arithmetic is the point. An agent proposes dependency upgrades and a team reviews them weekly.

Invented inputs. Acceptance = accepted / reviewed; revert rate = reverted / accepted; minutes per accepted = review minutes / accepted.

| Week | Reviewed | Accepted | Reverted | Review minutes | Acceptance | Revert rate | Minutes per accepted |
|---|---|---|---|---|---|---|---|
| 1 | 200 | 140 | 7 | 1,400 | 70% | 5.0% | 10.0 |
| 2 | 180 | 126 | 6 | 1,260 | 70% | 4.8% | 10.0 |
| 3 | 210 | 147 | 9 | 1,470 | 70% | 6.1% | 10.0 |
| 4 | 190 | 114 | 10 | 684 | 60% | 8.8% | 6.0 |

Week 3 crosses the revert threshold (6.1% against 5%), so the team samples 59 of the 147 accepted changes. Week 4 crosses a second: minutes per accepted change fell 40%, from 10.0 to 6.0, while the revert rate rose to 8.8%. Faster review with more reverts is the signature of approval turning into a click. The action is to stop widening autonomy, re-read the 10 reverted changes for what the reviewer missed, and sample 59 of the 114 accepted. Acceptance fell from 70% to 60% in week 4, a drop of 10 points. The threshold needs two weeks, so it has not yet fired: it fires if week 5 is also at or below 60%. Until then the drop is a prompt to check whether the model or the inputs changed, and to rerun the eval set early if they did.

## A real series: Rival's safeguard block rate

The safeguard block rate is the metric with a cause outside the team. Rival's post of 12 Nov 2026 says its package-analysis agents, which run on gpt-6.1-sol (a current model ID on [OpenAI's model page](https://developers.openai.com/api/docs/models/gpt-6.1-sol), read 19 Nov 2026), "normally" see about one thread in twenty blocked by the model's cyber safeguard, and that the rate rose to one in five on 5 Nov. Its chart gives the share of threads blocked, by day the thread started, from production traces of 27 Oct to 9 Nov 2026 (low-volume days left out).

Rival Security (vendor report, 12 Nov 2026), share of agent threads that hit the cyber block on gpt-6.1-sol; read 19 Nov 2026. 30 and 31 Oct and 7 Nov are absent because Rival's chart leaves out low-volume days. The threshold column is this page's rule applied to Rival's figures.

| Day (2026) | Threads blocked | Against a doubled 5% baseline (10%) |
|---|---|---|
| 27 Oct | 5.8% | Below |
| 28 Oct | 5% | Below |
| 29 Oct | 4% | Below |
| 1 Nov | 2.9% | Below |
| 2 Nov | 4% | Below |
| 3 Nov | 7.6% | Below |
| 4 Nov | 6% | Below |
| 5 Nov | 19.2% | Alert fires: first day above |
| 6 Nov | 20.8% | Above |
| 8 Nov | 7.7% | Below |
| 9 Nov | 18.5% | Alert fires again |

Three of Rival's findings change how to read the metric, from its replays on 10 Nov 2026. Blocks vary by workload: across package-analysis threads of 27 Oct to 10 Nov, network-facing packages were blocked 16.1% of the time against 6.2% for the rest, so compare like with like. Blocks are not fixed: of 296 unchanged replays of 78 blocked requests, 115 (39%) were blocked again, 33 of the 78 were never blocked again, and with three retries 84% of blocked requests got through. And a block lands mid-reply, a median of 27 seconds in. So record blocks by task type, count retried successes separately, and keep a fallback route. These are Rival's measurements on one provider's model, not general rates.

## The record kept at each review

AGENT REVIEW RECORD
Agent and version: ____ (model, prompt, tool list versions)
Review date: ____   Window: ____ to ____   Reviewer: ____
Proposals reviewed: ____  Accepted: ____  Reverted: ____  Review minutes: ____
Runs: ____  Escalations: ____  Threads: ____  Blocked: ____
Model spend: ____  Tasks completed: ____
Six metrics vs baseline: ____
Thresholds crossed: ____
Sample drawn (n and method): ____  Wrong items found: ____
Decision: continue / pause widening / rerun evals / roll back / switch route
Owner of the action and date due: ____
Change in model, prompt or tools since last review: ____ (counts as a new version)

Name the owner and schedule before go-live. A weekly review suits most jobs. Keep the records with the run logs described in [what to keep in an agent run log](/guides/agent-run-logs-what-to-keep/); the stop and rollback controls the decision depends on are on [kill switch and rollback](/evaluation-checklist/kill-switch-and-rollback/).

## Sources

- [UK NCSC, Guidelines for secure AI system development, secure operation and maintenance](https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-operation-maintenance), read 19 Nov 2026
- [NIST AI RMF 1.0 (NIST AI 100-1), MANAGE 4.1](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf), read 19 Nov 2026
- [Rival, When OpenAI's Cyber Classifier Flags Security Agents (12 Nov 2026)](https://rival.security/blog/openai-cyber-classifier-security-agents?utm_source=agenticaiincybersecurity.com&utm_medium=referral&utm_campaign=guide), read 19 Nov 2026
- [OpenAI, gpt-6.1-sol model page](https://developers.openai.com/api/docs/models/gpt-6.1-sol), read 19 Nov 2026

Last reviewed 19 Nov 2026

Canonical: https://agenticaiincybersecurity.com/guides/measuring-an-agent-after-it-ships/
