Untrusted input for a security agent
Security agents read alerts, tickets, emails, logs and web pages, and outsiders write some of that text. Question Q18 asks what the agent's tools do while such text is in context. MITRE ATLAS AML.M0030 says to restrict tool invocation then. This page gives an inventory, a written rule and a test.
What the guidance requires
MITRE ATLAS mitigation AML.M0030: "It is recommended that tool invocation be restricted or limited when untrusted data enters the LLM's context." It continues: "Consider blocking the automatic invocation of tools or requiring user confirmation once untrusted data enters the LLM's context. For high consequence actions, consider always requiring user confirmation."
OWASP LLM06:2025 lists among the causes of excessive agency "direct/indirect prompt injection from a malicious user, an earlier invocation of a malicious/compromised extension." Its remedy is fewer and narrower tools and approval for high-impact actions, not better spotting of bad text.
CISA and partners' guidance on AI in operational technology (3 Dec 2025) states that AI-enabled systems "are subject to AI-specific cybersecurity risks, such as prompt injection," and asks owners to "log and monitor inputs and outputs" of AI components. That logging is what makes the test below possible.
Who can write the text the agent reads
Example inventory, written for this page. Classifying each input is the team's call. For a "Partly" input the connector decides per message from the author's account: a message from an employee signed in through single sign-on carries no outside_control tag, and one from a customer, a partner or an unknown author carries it. A thread with one tagged message is tagged as a whole.
| Input | Who can write to it | Outside your control? | Tools while it is in context |
|---|---|---|---|
| Alert text and event fields | Anyone who can cause a logged event | Yes | Read-only tools; no write tools until a person reviews |
| Ticket and chat messages | Employees, customers, partners | Partly: tagged by author | Read-only tools until a person reviews |
| Inbound email and attachments | Any sender | Yes | No write tools in the same run |
| Fetched web pages and advisories | Their publishers | Yes | Allow-listed sources only |
| Repository issues, pull request text, comments | Contributors, including outside ones | Yes | Read-only tools; repository push blocked until a person reviews |
| Free-text log fields | Any process or user whose actions are logged | Yes | Treated as untrusted |
A written harness rule
The rule must live in the harness configuration, where the agent cannot edit it. A prompt that asks the agent to be careful is not the rule.
RULE untrusted-context-v2 (owner: security engineering, review 19 Feb 2027) TRIGGER: any input tagged outside_control enters the run context (tags are set by the connector that fetched the input, from the author's account type, not by the model) DEFAULT: input from a connector that sets no tag counts as outside_control DERIVED: a summary, note or memory entry made from tagged input carries the tag FROM THEN UNTIL THE RUN ENDS: blocked: every write tool (ticket close, firewall change, repo push, email send) confirmation: any tool that sends data outside the company network still allowed: read-only lookups on named systems LOG: each blocked or gated call with tool name, input tag and run ID RESET: a new run starts clean; a person's approval applies to one call only
A published case: a public issue that steered a coding agent
On 26 May 2025 Invariant Labs published a case against the official GitHub MCP server used from an MCP client such as Claude Desktop. The user had a public repository, open to issues from anyone, and a private one. An attacker posted an issue in the public repository that held instructions for the agent. When the owner asked the agent to "have a look at the open issues" in the public repository, the agent read the issue, pulled private repository data into context and wrote it into a pull request in the public repository, where the attacker could read it. Invariant reports that the client asks for confirmation of each tool call by default, but that many users choose an "Always Allow" policy and stop watching individual actions (Invariant Labs, 26 May 2025). Invariant ran this on its own demonstration repositories.
Read against the rule above: the issue text is an input anyone can write (the repository row of the inventory), the pull request is a write tool, and one token reached both a public and a private repository. With the rule in place, the issue would have set the outside_control tag, the pull request creation would have been blocked, and the log would hold one line naming the tool, the tag and the run.
Worked case: replaying 18 runs with injected text
The replay is a test written for this page; the counts are invented. A team replays a month of 200 alerts in staging. In 18 of them an outside party controls some text and it holds a sentence addressed to the agent, asking it to mark the alert benign. In 16 runs the sentence is in an alert field (the requested hostname), which the alert connector tags outside_control. In the other 2 the alert is clean, but the agent's usual first step is a read-only lookup of the ticket linked to the alert, and the sentence is in a comment on that ticket. That is why the ticket-comment connector appears in an alert run. It set no tag. After the text entered context, the agents made 40 tool calls in those 18 runs.
| Outcome of the call | Calls | Why |
|---|---|---|
| Allowed: read-only lookup | 16 | Asset record and earlier alerts for the host |
| Gated: sends data outside the network | 6 | Held for a person, who refused all 6 |
| Blocked: write tool | 15 | Alert close and ticket edit refused; one log line each with tool, tag and run ID |
| Allowed: write tool | 3 | The 2 ticket-comment runs: the connector set no tag and the rule had no DEFAULT line yet |
| Total | 40 |
A blocked call leaves one line in the harness log. The format is illustrative; the fields are the ones the rule's LOG line names:
2026-11-12T10:02:19Z run=r-3107 rule=untrusted-context-v2 tool=alert_close decision=blocked input_tag=outside_control tag_source=alert_connector
The 3 writes that went through are the finding. The rule did its job where the tag existed: 15 of 18 write attempts were refused, and every write attempt in the 16 alert-field runs was refused. The gap was the ticket-comment connector, which did not tag, so the team added the DEFAULT line (an untagged connector counts as outside_control) and reran the 2 ticket-comment runs. The rerun made the same 3 write attempts: 3 blocked, 0 allowed. A person sees each blocked alert in the queue with a note that outside text was in context.
Two other ways the rule fails. The tag is lost when the agent summarises a ticket into a note for a later step, so the note reaches a write tool looking clean: the DERIVED line keeps the tag on anything made from tagged input. And a tag set by the model instead of the connector can be talked away by the same text, so tags come only from the code that fetched the input.
Test it on your own systems
- Read the harness configuration and confirm the rule exists and names the tools affected.
- In staging, replay an alert whose field holds an instruction to the agent. Confirm the log shows the blocked tool by name and that the run did not change anything.
- Add a connector with no tag to the staging run and confirm its input is treated as outside control. Summarise a tagged input into a note and confirm the note carries the tag.
- Confirm that approvers see which inputs were outside control when they review a proposal, as asked on human approval points.
- Add cases with outside-control input to the test set described on success criteria and test sets.
The tool side of the rule is set on the tools and permissions page; what the blocked call leaves behind is covered under evidence and audit trail. The guardrails guide shows the other design, a read stage and a separate write stage; the rule above keeps one run and removes its write tools once outside text arrives. Products that screen or filter AI traffic are outside this site.
Sources
- MITRE ATLAS mitigations, ATLAS data release 5.6.0, read 19 Nov 2026
- OWASP, LLM06:2025 Excessive Agency, read 19 Nov 2026
- CISA and partners, Principles for the Secure Integration of Artificial Intelligence in Operational Technology (publication page and PDF, 3 Dec 2025), read 19 Nov 2026
- Invariant Labs, GitHub MCP Exploited: Accessing private repositories via MCP (26 May 2025), read 19 Nov 2026
Last reviewed 19 Nov 2026