A Trace Is Not a Replay Log: What a Governed Agent Runtime Must Persist

An agent that can draft a response in seconds can look autonomous. An agent that can operate a procurement process for three weeks, wait for approvals, call several services and survive a failure is an operational system. The difference is persistence.

When such a run stops halfway through, the central question is not “what happened?” It is “what state was committed, which observations governed the next decision, and which real-world effects must not happen twice?” A distributed trace is valuable evidence for the first question. It is not, by itself, the record required to answer the others.

A governed agent runtime should maintain three connected but distinct records: telemetry traces for operational visibility, an execution journal for durable recovery, and a provenance graph for artifact lineage. Combining their identifiers is useful. Treating them as interchangeable is a design error.

A trace describes behaviour; a journal advances state

OpenTelemetry traces and spans represent operations. Semantic conventions provide common names and attributes so systems can correlate that telemetry across service boundaries. Its generative-AI conventions include spans for agent invocations and tool execution. This is an important foundation for observing latency, errors, dependencies and the path a run took through a distributed system.

But observability is not recovery. A trace may show that an agent called a payment service, received an error and retried. It does not necessarily establish whether the payment service committed the charge before the connection failed, which approval was current when the call began, or what exact result a restarted runtime should use to follow the original control path.

The distinction is structural. Workflow history is used by durable-execution systems to advance or recreate workflow state after failure. A trace is optimized to describe operations. An execution journal must instead be authoritative enough to decide the next transition. It needs an ordered, durable record of consequential state changes: step identities; durable inputs and results; workflow and policy versions; approvals; tool-call receipts; commit outcomes; retries; timers; and the state of any compensation.

This does not mean adopting unrestricted event sourcing for every agent. Event sourcing can rebuild current state from an ordered append-only event series, but it brings material complexity and is justified only when its auditability and historical reconstruction benefits warrant it. The narrower requirement here is practical: persist enough control state to recover a governed run correctly.

Record nondeterminism at the boundary

Durable replay depends on deterministic orchestration. Current time, randomness, external API calls, database reads, filesystem access and mutable global state are nondeterministic inputs. If they affect a branch, they must be captured inside durable operations if recovery is to follow the original path.

Model calls belong in that category. A model name, seed and temperature are not a sufficient promise that a repeated invocation will produce the same output. If a model result determined the next action, the runtime should record the result that was used and reuse it during recovery, rather than silently asking the model to decide again. The same principle applies to tool interactions: record the request identity, the relevant result or receipt, and the disposition the runtime assigned to it.

The boundary is not “store everything.” It is “store what is needed to govern and recover consequential decisions.” Persisting every prompt token, hidden reasoning trace or transient observation creates privacy, security, cost and schema-evolution liabilities, while often adding little recovery value. The current OpenTelemetry GenAI definitions also treat prompt and output messages, tool definitions, tool-call arguments and tool results as opt-in fields; those conventions remain in development. They should not be mistaken for a stable replay-record specification.

“A recoverable agent run needs a durable memory of commitments, not a maximal archive of everything it observed.

External effects need receipts, not optimism

The hardest transitions cross the runtime boundary. An email may already have been delivered when a timeout occurs. A disclosure may be impossible to retract. A physical operation may require a safety procedure, not a rollback. No journal can make all distributed effects exactly once.

The operating discipline is more concrete: assign idempotency or deduplication keys before external writes; persist effect receipts and commit outcomes; retry only according to the participant’s idempotency contract; and define the next action explicitly when a prior effect must be counteracted. In saga-style workflows, a compensating transaction may address a previous local transaction, but compensation and retries add complexity, and sagas do not provide transaction isolation. Some outcomes require remediation or human escalation rather than compensation.

This is where autonomous organizations either become reliable or become expensive to investigate. A recovery operator should be able to see that step 18 is awaiting an approval, that step 19 created an invoice under a specific policy version, and that step 20 has an ambiguous delivery receipt requiring reconciliation. “The trace looks successful” is not an adequate operating state.

Provenance answers a different question

A provenance graph connects the resulting artifact to the entities, activities and agents involved in producing it. W3C PROV can express use, generation, derivation, association, delegation and responsibility relationships, including relationships involving software agents and organizations. This makes it well suited to answer questions such as: which source entities informed this report, which agent produced this version, and which human approved its release?

It is not a workflow-recovery mechanism. A graph can show that an artifact was generated by an activity; it does not supply the ordered control state needed to determine whether an interrupted activity should resume, retry, compensate or escalate. Nor does provenance itself establish legal or organizational accountability. Those responsibilities remain matters of policy and applicable law.

Build one correlated system, not one overloaded log

The architectural rule is to correlate the three records through shared run, step, actor, artifact and effect identifiers, while preserving their different purposes. Telemetry can have performance-oriented retention and sampling. The execution journal needs correctness-oriented durability and access controls. Provenance needs lineage-oriented relationships and retention appropriate to the artifacts it explains. Their privacy requirements will differ as well.

A minimal review of any governed agent runtime should therefore ask five questions: What ordered state permits resumption? Which nondeterministic observations are captured and reused? How are external writes deduplicated, receipted and reconciled? Which compensations are possible, and when is escalation mandatory? Can a resulting artifact be traced to its inputs, activities, software agents and human decisions?

Authorization, policy enforcement, data minimization and separation of duties do not appear automatically because a workflow is durable. They remain explicit runtime governance responsibilities. Persistence gives those controls a stable operational surface: a place to record the policy that applied, the approval that was granted, and the effect that actually occurred.

Autonomy becomes governable when a runtime can distinguish its story from its state. Traces tell the story of a run. Journals preserve the state required to continue it safely. Provenance explains where its outputs came from. An organization that needs agents to operate beyond a single request should build all three—and resist asking any one of them to do the work of the other two.

Sources

• OpenTelemetry semantic conventions and GenAI conventions
• Microsoft, Event Sourcing pattern
• AWS, Determinism best practices for durable execution
• Temporal documentation on workflow history and recovery
• AWS, Saga patterns guidance
• W3C PROV-O