All field notes

Governed Runtime

Two Agents Do Not Make Two-Person Control

Multiple agent approvals are meaningful only when the approving agents are independent across the failure domains that matter to a business effect.

MP
Max PerfiljevFounder & CEO, AES · Architect of Autonomous Organizations
Read in Russian

A proposed payment, deployment, customer-data export or contractual commitment may soon pass through several AI agents before it is executed. One agent prepares the action; others review the evidence, challenge the reasoning or approve the result. On a dashboard, this resembles maker-checker control. In practice, it can be one mistake repeated several times.

Two agents do not make two-person control. Agent count is not a measure of independent judgment. If the proposer and reviewers use the same model family, receive equivalent prompts, retrieve from the same knowledge base, inherit the same erroneous memory, operate under one team, or call through the same toolchain and credentials, their approvals may belong to one failure domain. A quorum built this way can produce more records without producing a stronger decision.

Autonomous organizations should treat approval independence as a runtime property to be tested against the exact business effect. The relevant question is not “did two named agents approve?” It is “did this effect receive the required approvals from principals sufficiently separated for this risk class?” That distinction turns a familiar control into an architectural primitive: an independence-aware approval quorum.

Separation of duties is broader than identity

NIST SP 800-53 AC-5 requires organizations to identify duties that need separation and define access authorizations that support it. Its discussion makes an important architectural point: separation-of-duties violations can span systems and application domains. A control cannot be assessed only by looking at isolated identities in one application.

That principle applies directly to agentic operations. Distinct agent IDs and valid signatures establish attribution and integrity. They show which principal submitted an approval and whether its record was altered. They do not establish that two approvals arose from independent reasoning, independent evidence or independently governed infrastructure.

NIST's dual-authorization control, AU-9(5), describes approval by two authorized individuals for specified audit operations and suggests rotating duties to reduce collusion risk. It is a useful pattern, but it is not an existing requirement that two AI agents satisfy two-person control. Applying the pattern to mixed human-agent organizations is an architectural extension. It must be explicit about what is being separated and why.

The approval must bind to an effect, not an intention

An approval should be a signed, durable assertion over a precise approval subject. At minimum, that subject should identify the proposed business effect, the evidence set used to assess it, and the policy version under which it was reviewed. “Approve the vendor payment” is too vague. “Approve transfer of this amount, to this counterparty, from this account, under these constraints, supported by this evidence” is reviewable and enforceable.

This binding prevents approval laundering: reusing a valid-looking approval after the action beneath it has changed. If the destination, amount, scope, evidence, or governing policy changes, the approval must become stale. GitHub protected branches provide a useful precedent. Reviews can be dismissed when the reviewed diff changes, and a recent reviewable push can require approval from someone other than its author. An autonomous runtime needs the same discipline for business effects, not only source changes.

Freshness does not mean that every harmless presentation change should trigger a full review. The runtime needs a canonical representation of the effect and a policy-defined account of material inputs. But it must be able to demonstrate exactly which version of an action each approver saw. Otherwise an approval is a comment on a past proposal, not authorization for the current one.

Make the approval carry its independence evidence

Every approval should carry an attestation describing the approver and its operating context. The goal is not to produce a universal numerical independence score. The goal is to give policy enough structured evidence to reject correlated approvals when the consequence demands it.

  • Approver principal and credential: the accountable identity, authorization path and signing credential.
  • Operator and ownership boundary: the human or organizational team responsible for the agent’s configuration and operation.
  • Model lineage: the deployed model, relevant family or provider, and any policy-defined lineage grouping.
  • Prompt, memory and knowledge sources: the instructions, persistent memory and retrieved evidence that could create shared assumptions or shared contamination.
  • Tools and credentials: external systems, delegated permissions and control paths used during review.
  • Runtime domain: the infrastructure, service boundary or other operational domain whose failure could affect multiple approvers.

The attestation need not reveal sensitive prompts, proprietary model internals or all retrieved content. It must, however, provide stable references that a policy engine can compare. A reviewer might attest to a knowledge-source set identifier rather than its documents, or to a runtime-domain label rather than detailed topology. The design problem is evidence sufficient for enforcement and later audit, with disclosure appropriate to the environment.

A quorum is a predicate, not a headcount

The policy engine should evaluate a risk-specific independence predicate over a set of fresh approvals. A low-impact internal action may require distinct principals and prohibit proposer self-approval. A payment or production change may additionally require a reviewer outside the proposer’s operator boundary. A high-consequence export might require separation across selected data, tool, model or infrastructure domains, plus a human approval. These are examples, not universal thresholds.

The predicate should state both the required number of approvals and the separation dimensions that count. For example: accept one proposer and two approvers only if none is the proposer, each uses a distinct credential, at least one approver is operated by a separate team, and the two approvers do not share the designated knowledge-source or runtime domain. The runtime can then return a concrete denial reason: sufficient approvals exist, but they fail the required independence relation.

This is more useful than assigning every agent an abstract independence rating. Independence is contextual. Sharing a model provider may be acceptable for a routine scheduling change and unacceptable for a regulated funds movement. Conversely, different providers are not proof of independence. A large empirical study of more than 350 language models found substantial correlation in model errors; shared architectures and providers contributed, but distinct providers or architectures did not guarantee uncorrelated errors. Model diversity can reduce some common-mode failure, not eliminate it.

Design for common-mode failure, then operate it

NIST SP 800-53 SC-29 makes the same balanced point about technology diversity: it can reduce common-mode failures, but it adds complexity and management overhead. An independence-aware quorum should therefore be selective. Forcing different models, teams, stores and infrastructure onto every low-risk operation creates cost and fragility without a corresponding control benefit. Risk classification should decide which correlated paths are intolerable.

Operationally, this requires maintaining an inventory of relationships, not merely a registry of agents. When a team changes an agent’s model, moves its retrieval service, grants it a shared tool credential, or takes over its operation, the attestation data and policy evaluation must reflect that change. Otherwise the organization is enforcing a diagram from last quarter while the actual failure domains have converged.

Ownership matters as much as technology. AWS security architecture guidance recommends dividing independent controls among centralized teams that peer-review one another’s policies. The lesson is not that a particular team chart guarantees safety. It is that organizational boundaries are part of the control design. A fleet of technically varied agents administered, prompted and corrected by one operator group may still offer a thin separation of duties.

What the governed runtime must decide

At commit time, before an irreversible or consequential effect, the runtime should resolve the effect version, policy version and required quorum. It should verify each approval’s integrity, authorization, freshness and scope. It should evaluate the independence predicate over the attestations, record the decision and reject the effect if the predicate is unsatisfied. A later audit should be able to reconstruct both the approvals and the relationship data on which the decision relied.

The outcome is not a claim that autonomous organizations have recreated human judgment by multiplying agents. It is more concrete: the organization can distinguish multiple confirmations from genuinely separated control paths, and can demand more separation when an effect warrants it. That is the standard an approval quorum should meet. Otherwise, it is only a counter that happens to count agents.

BUILD WITH AES

Turn architecture into an operating company.

AES connects strategy, tasks, organizational memory, knowledge, agents, people and approvals in one execution environment.