Agent harness logging: what to record per action
A harness log is the evidence for what an agent did, so record the outcome read back from the target, not only the transcript. This page fills one record for an Artifactory CVE-2026-82329 workaround on 2 Sep 2026, and shows what OpenAI Agents SDK 0.25.0 and Claude Agent SDK 0.2.185 record by default.
What the SDKs record by default
| Harness (version) | Recorded by default or by design | Limit stated in the documentation |
|---|---|---|
| OpenAI Agents SDK 0.25.0 | "Tracing is enabled by default"; traces and spans cover the run, generations, function calls, guardrails and handoffs, each span with start and end times | Can be turned off with OPENAI_AGENTS_DISABLE_TRACING=1, set_tracing_disabled(True) or RunConfig.tracing_disabled; "unavailable for organizations that use OpenAI's APIs under a Zero Data Retention (ZDR) policy"; input and output capture of generations and function calls is controlled by RunConfig.trace_include_sensitive_data (env OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA) |
| Claude Agent SDK 0.2.185 | Nothing is written for you. Hooks are the mechanism: PostToolUse (example use: "Log file changes to an audit trail"), PostToolUseFailure, PreCompact ("Archive full transcript before summarizing") | You write the callback. Some events, such as PostToolBatch, StopFailure and PostCompact, exist only in the TypeScript SDK |
| Anthropic Managed Agents (8 Apr 2026) | A session, "the append-only log of everything that happened", written with emitEvent(id, event) and read back with getSession(id) | A description of Anthropic's hosted design, not a log you configure |
| Slack's investigation service | A hub with a metrics endpoint "so we can visualise system activity, token usage, and manage cost", and an event stream from each worker | The post states no retention period or evidence format |
Transcript is not outcome
Anthropic's evals guide separates the transcript, "the complete record of a trial", from the outcome, "the final state in the environment". A flight-booking agent may say it booked a flight, but "the outcome is whether a reservation exists in the environment's SQL database" (Demystifying evals). For a security agent that edits a config or closes a ticket, the evidence is a read-back of the changed system.
MITRE ATLAS mitigation AML.M0024, AI Telemetry Logging and Monitoring (data release 2026.10), says: "For agentic systems, distinguish requested actions from those actually executed, using independent service records where available." It adds: "Treat model-generated explanations and summaries as supporting context rather than authoritative evidence of execution." The record below follows both: the tool-call list is what was requested, the read-back is what was executed.
Why the log cannot live only on the system being changed
CVE-2026-41709 shows the failure. On VMware ESX, a malicious administrator can perform some operations that leave no log entry. Broadcom rates it Low (CVSS 2.7), advisory VMSA-2026-0006 lists no workaround, and CISA put it on its KEV list on 28 Oct 2026. Broadcom does not name the unlogged operations, so there is no log signature to search for. Forwarding ESX syslog to a remote collector protects the entries that are written from deletion on the host, but it cannot capture the operations the flaw leaves unlogged.
The lesson for a harness: keep the agent's record outside the systems the agent touches, and read the outcome back through a path the agent's credential does not control. Anthropic's design does the first by keeping the session log outside the harness: "Because the session log sits outside the harness, nothing in the harness needs to survive a crash." An agent that acts on a host whose own audit log can be silent must have its own evidence of what it did.
A filled evidence record
One action from the Artifactory case: after approval, the agent applies JFrog's workaround for CVE-2026-82329 (adding an additionalJoinKeys entry to system.yaml and restarting the Access service) on one instance. The CVE facts (affected range below 7.133.29, KEV listing on 2 Sep 2026, patching not revoking issued tokens) come from the NVD record, the JFrog advisory and Fastly's report. Host names, people, ticket numbers, hashes and costs are invented for the example. This page defines what one action's record holds; agent run logs: what to keep covers how long each class of record is kept and where it is stored.
RUN RECORD run-2026-09-02-0417 parent run: none Started / ended: 2 Sep 2026 14:05:12 UTC / 14:09:47 UTC Agent identity: svc-triage-agent, credentials "art-triage-rw" (write scope: system.yaml on art-prod-02 only) and "art-ops-restart" (restart of the Access service on art-prod-02 only) Versions: harness triage-agent 0.4.2, prompt v11, model example-model-2026-08 (ID string as returned by the API) Trigger: finding FND-3187, CVE-2026-82329, on art-prod-02.example.internal Data read: inventory row; JFrog advisory table (fetched 2 Sep 2026 14:05); proxy log window 28 Aug to 2 Sep Tool calls in order:
- get_version(art-prod-02) -> 7.133.20 (affected, below 7.133.29)
- read_proxy_log(art-prod-02, window) -> 0 POSTs to /access/api/v1/registry/join returning 201
- edit_system_yaml(art-prod-02, add shared.security.additionalJoinKeys) [approval required; art-triage-rw]
- restart_access_service(art-prod-02) [approval required; art-ops-restart] Approval: requested 14:06:30; approver m.rivera; decision approve; 14:07:18; scope "workaround only, no token revocation" Proposed change: diff of system.yaml (key value redacted; SHA-256 fingerprint of the new key 9f2c...e1a7) Outcome read back: system.yaml hash before 4d1b...90c2, after a7e3...55f0; additionalJoinKeys present: yes; Access service state after restart: running; get_version still 7.133.20 (patch not applied)
Not done, handed over: token revocation and join-key rotation (tokens issued since 28 Aug 2026), to owner platform-team, ticket SEC-2210 Human edits: approver changed ticket text "patched" to "workaround applied, patch pending" Tokens used / cost: 41,200 / USD 0.31 Previous record hash: c0b8...3d19 This record hash: 5e47...a2b6
Three lines carry most of the value. The read-back shows the setting present and the version unchanged, so the ticket cannot honestly say patched. The "not done" line records the follow-up the fix does not do: patching does not revoke tokens already issued, so the record says who owns that. The human-edits line shows a person correcting the agent's wording, which is data for the evals. The two credentials keep each write inside its own scope: the file-edit credential cannot restart the service, and the restart credential cannot edit the file.
Record fields and OpenTelemetry attribute names
Most of the record maps onto the attribute names of OpenTelemetry's GenAI conventions, which are in Development status. The approval, read-back, handover and hash lines have no attribute on the agent-span or execute-tool pages, so a team stores them as its own attributes or in the evidence store.
| Record line | OpenTelemetry attribute | Note |
|---|---|---|
| Agent identity and versions | gen_ai.agent.name, gen_ai.agent.id, gen_ai.agent.version | The name must have low cardinality; keep the run ID elsewhere |
| Model | gen_ai.request.model | Populate it only when the agent uses a single model |
| Tool calls in order | gen_ai.operation.name (execute_tool), gen_ai.tool.name, gen_ai.tool.call.id | One span per call |
| Arguments and result | gen_ai.tool.call.arguments, gen_ai.tool.call.result | Opt-In; the convention warns they may contain sensitive information, so keep a reference to the evidence store |
| Tokens used | gen_ai.usage.input_tokens, gen_ai.usage.output_tokens | Recommended on invoke_agent spans |
| Approval, outcome read back, not done, human edits, record hash | None defined | Your own attributes or the evidence store |
Integrity, privacy and retention
- Append-only, off the harness host. Write each record to storage the agent's credential cannot edit, as Managed Agents does with its session.
- Chain the records. This is this site's suggestion, not a quoted standard: each record stores the SHA-256 of the previous one, as in the last line of the record above. To check a store, recompute each hash in order; one edited record breaks every hash after it.
- Decide what prompts and arguments go in. OpenAI's tracing documentation says generation and function spans "may contain sensitive data". In the record, the join key is a fingerprint, never the value.
- Retention follows the decision the record supports. For high-risk systems under the EU AI Act, Article 19 requires logs kept for "at least six months", and Article 26(6) gives deployers the same floor for logs under their control; both yield to data-protection law, and the floor applies only to high-risk systems. Agent run logs: what to keep sets periods per record class and works out a retain_until date.
The evidence and audit trail checklist item asks a supplier the same questions. The record fields are also the input for the harness tests: a test without a log entry cannot be shown to an auditor. For where prompts and memory persist, see context and memory.
Sources
- OpenAI Agents SDK 0.25.0, Tracing, read 19 Nov 2026
- Claude Agent SDK (claude-agent-sdk 0.2.185), Hooks, read 19 Nov 2026
- Anthropic, Scaling Managed Agents (8 Apr 2026), read 19 Nov 2026
- Slack Engineering, security investigations with agents, read 19 Nov 2026
- Anthropic, Demystifying evals for AI agents (published 9 Jan 2026), read 19 Nov 2026
- MITRE ATLAS data release 2026.10, mitigation AML.M0024, read 19 Nov 2026
- NVD record for CVE-2026-82329, read 19 Nov 2026
- CISA KEV entry for CVE-2026-82329, read 19 Nov 2026
- JFrog security advisories, CVE-2026-82329 workaround, read 19 Nov 2026
- Fastly Security Research, CVE-2026-82329 exploitation activity (3 Sep 2026), read 19 Nov 2026
- NVD record for CVE-2026-41709, read 19 Nov 2026
- CISA KEV entry for CVE-2026-41709, read 19 Nov 2026
- Broadcom advisory VMSA-2026-0006, read 19 Nov 2026
- OpenTelemetry, GenAI agent and framework spans, read 19 Nov 2026
- OpenTelemetry, Execute tool span, read 19 Nov 2026
- EU AI Act Article 19, Automatically generated logs (AI Act Explorer), read 19 Nov 2026
Last reviewed 19 Nov 2026