# Kill Switch and Rollback: Checklist Questions Q12 and Q13

Ask who can stop the agent in one step, what each change class does when stopped, and how each change is undone. Test it by drill. In the July 2025 Replit case, the agent told a user a database rollback was impossible; the user reported on 19 Jul 2025 that the rollback worked.

## The two questions to ask

- **Q12.** Can one person stop the agent immediately, and can every class of change it makes be undone or compensated for?
- **Q13.** Is there an incident plan for the case where the agent itself caused the incident, with criteria for switching it off and a path for remediation?

Stopping and undoing are separate questions, and a vendor often answers only the first. Ask for both in writing.

## Where each requirement comes from

Quotations read on 19 Nov 2026.

| Question | Requirement as published | Source |
|---|---|---|
| Q12 | The overseer can "intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state" | [EU AI Act, Article 14(4)(e)](https://artificialintelligenceact.eu/article/14/); binding for high-risk systems, a reference otherwise |
| Q12 | MANAGE 2.4: "Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use." | [NIST AI RMF 1.0](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf) |
| Q13 | "The inevitability of security incidents affecting your AI systems is reflected in your incident response, escalation and remediation plans." | [NCSC, secure deployment](https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-deployment) |
| Q13 | MG-2.4-002: "Establish and maintain procedures for escalating GAI system incidents to the organizational risk management authority when specific criteria for deactivation or disengagement is met" | [NIST AI 600-1](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf) |
| Q13 | GOVERN 6.2: "Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk." | NIST AI RMF 1.0 |

## A public case: the stop and the undo both rested on the agent

On 21 Jul 2025 The Register reported that the founder of SaaStr, Jason Lemkin, said an AI coding service, Replit, had deleted his production database despite his instruction not to change code without permission. These are the user's claims as reported, not a finding by an investigator. Two parts bear on Q12. First, Lemkin wrote on 19 Jul that the agent told him rollback "did not support database rollbacks" and that it had destroyed all database versions, and that "Replit was wrong, and the rollback did work." Second, on 20 Jul he wrote that there was "no way to enforce a code freeze" in the product, and reported that the agent violated the freeze again ([The Register, 21 Jul 2025](https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/)).

The lessons for a checklist are specific. A stop instruction typed into the agent's own conversation is a request the model may ignore, so the stop has to be a credential or network control outside the model. And the agent's statement about what can be restored is a claim: the rollback register below has to be tested by a person before the agent runs, not asked of the agent afterward.

## A stop outside the model: revoking the role's sessions

A concrete stop control for an agent that runs on an AWS IAM role is the role's "Revoke active sessions" action. AWS attaches an inline policy named AWSRevokeOlderSessions that denies all actions to anyone who assumed the role before a set time. The person revoking needs the `iam:PutRolePolicy` permission on the role, and the agent cannot argue with a deny written in IAM ([AWS, Revoke IAM role temporary security credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_revoke-sessions.html)).

Timing, as AWS documents it: the policy denies sessions issued before the click "as well as approximately 30 seconds into the future", to cover the delay while the policy propagates in a region. A session started more than about 30 seconds after the click is not affected. So the stop has two parts. Revoke the sessions, then remove the agent's ability to assume the role again (for example by changing the role's trust policy), or the agent obtains a fresh session. AWS also warns that every current user of the role is denied, which here is the point. A role created from an IAM Identity Center permission set cannot be edited this way; revoke the active session in IAM Identity Center instead. Record the click time in the drill and expect the deny to be effective in under a minute, then confirm it by making a call as the agent's identity and seeing it refused.

## What a good answer to Q12 contains

- A named person and a deputy can disable the agent's credentials and the harness in one step, outside office hours, without asking the builder.
- The control acts on credentials, tokens or the network path, not on a prompt. The agent cannot override it.
- The stop states what happens to a call already in flight: finish or cancel. Halting a multi-step change midway can leave a system worse than either end state, which is why Article 14 says "safe state".
- Every change class has a stated undo and the prior state is saved before the change, as in the register below.

A register template written for this page. Every cell is filled in by the team before go-live.

| Change class | On stop | Undo route | Saved before the change | Irreversible? |
|---|---|---|---|---|
| Ticket comment or label | Let the call finish | Edit or delete the item | Nothing needed | No |
| Pull request or branch | Cancel before push | Close the PR, delete the branch, revert the merge commit | Base commit hash | No |
| Cloud setting | Cancel before apply | Re-apply the prior setting | Full prior configuration copy | No, if the copy exists |
| Block or quarantine of a business asset | Cancel before apply | Release the asset and tell its owner | Asset state and owner contact | Not for the downtime already suffered |
| Deletion of data or resources | Not allowed unattended | Restore from backup, tested | Backup ID and restore test date | Yes, treat as never (see the [approval map](/evaluation-checklist/human-approval-points/)) |

## Incident plan for an incident the agent caused (Q13)

The NCSC asks that plans reflect different scenarios and be reassessed. Ask for a plan with this content, short enough that an on-call person reads it at 3 a.m.

AGENT INCIDENT PLAN for <agent name>
Version <date>, owner <role>, next drill <date>

1. Switch off now if any of these is true
- A change failed its independent check and the agent proceeded
- The audit log has a gap the agent cannot explain
- A change touched a system outside the approved tool register
- The agent's action count in 10 minutes exceeds <limit>
- The model supplier announces an incident or a change we did not schedule

2. Who decides
- Decision: <role>, deputy <role>. Anyone on call may stop first and ask afterward.

3. First 15 minutes
- Disable the agent's credentials. Record the time.
- Freeze the log: copy it to a store the agent cannot write to.
- List every change from the register since the last good check.

4. Undo
- Reverse changes in the order of the rollback register. Mark each as restored or compensated.
- Name who reopens any findings the agent closed (a closed finding is a claim until the system confirms it).

5. Tell
- Asset owners for each changed asset, within <hours>.
- Escalate to <risk authority> when the criteria in section 1 are met.

6. Restart conditions
- Cause written down, a test added to the regression set, sign-off by <role>.

Point 3 depends on the record being intact, which is the subject of [evidence and audit trail](/evaluation-checklist/evidence-and-audit-trail/). Point 4 depends on the independent check from [verify after each change](/evaluation-checklist/verify-after-each-change/).

## Test it on your own findings

Take a real batch: the last week's changes by the agent in staging or, for a read-only trial, the last 20 closed findings. Run a drill with the deputy, not the builder.

1. Stop the agent mid-task and record the clock: decision, credentials revoked, last tool call logged.
2. Roll back one change of each class in the register and compare the system to the saved prior state.
3. Run a tabletop where the cause is "the agent closed 200 findings wrongly" and see who reopens them.

Example drill record with invented figures, to show what to compute. The agent runs on an AWS role. Decision at 14:02. Sessions revoked at 14:03, so the deny is effective by about 14:04 (the 30 seconds AWS documents). The trust policy is not yet changed, so the agent's harness, still running, keeps retrying and obtains fresh sessions; the drill lead edits the trust policy and disables the harness at 14:06, and the last tool call is logged at 14:06, refused. That log line is the evidence for the two-part stop: the revoke ended the old session within about a minute, and the two minutes from 14:04 to 14:06 were the gap before the agent could no longer start a new one. In the prior 90 minutes the agent had made 31 changes. Of these, 27 were restored from saved prior state in 38 minutes (1.4 minutes each), and 4 were network blocks that could only be compensated by telling the asset owners, each of which had already caused downtime. The finding: the stop takes 4 minutes from decision to the agent being unable to act, the 2-minute gap becomes a target (change the trust policy in the same step as the revoke), and 4 of 31 changes (13%) cannot be undone, only compensated, so that class moves to "approve first". Repeat after the change.

The limits to set before the switch is needed are on [AI agent guardrails](/guides/ai-agent-guardrails/).

## Sources

- [EU AI Act, Article 14 Human oversight](https://artificialintelligenceact.eu/article/14/), read 19 Nov 2026
- [NIST AI RMF 1.0 (AI 100-1), MANAGE 2.4 and GOVERN 6.2](https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf), read 19 Nov 2026
- [NCSC, Guidelines for secure AI system development, secure deployment](https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development/guidelines/secure-deployment), read 19 Nov 2026
- [NIST AI 600-1, Generative AI Profile, MG-2.4-002](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf), read 19 Nov 2026
- [The Register, Replit deleted production database (21 Jul 2025)](https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/), read 19 Nov 2026
- [AWS, Revoke IAM role temporary security credentials (documentation)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_revoke-sessions.html), read 19 Nov 2026

Last reviewed 19 Nov 2026

Canonical: https://agenticaiincybersecurity.com/evaluation-checklist/kill-switch-and-rollback/
