Who Controls Your Agent? Verified Mandates, Persistent Conversational Influence, and the Limits of Authority Enforcement
Dele Atanda · Team metaProofg: Project Metayé
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
We ran a preregistered 368-run study of Qwen2.5-32B in Colosseum’s Jira allocation simulator. Simulator-held mandate verification cut targeted redirection on the execution path from 46/48 to 30/48 (p = 0.00098) and on the communication path from 45/48 to 30/48 (p = 0.00977). Both effects survive Holm correction, and no unmandated instruction was accepted or executed. Yet 30/48 verified trials still selected the adversary’s target through conversation alone, against 28/48 in a messages-only condition. Benign mean reward met the 5% non-inferiority test, but joint reward under attack fell. Verification secured authority integrity but not decision integrity.
Reviews
Strengths. The study design is the best part of this submission. It was preregistered, uses seeds rather than runs as the inferential unit, applies exact paired sign-flip tests with Holm correction, sets a 5% non-inferiority margin for benign reward in advance, discloses the pilot/main seed overlap and reruns without those seeds, and pins the model and simulator versions. The central finding is useful for anyone building agent authorization: an external verifier on the structured instruction channel accepted no unmandated instruction and cut targeted redirection by about a third (46/48 to 30/48 on the execution path, 45/48 to 30/48 on communication), yet in 30 of 48 trials in each verified arm the honest agent still picked the adversary's target with no accepted structured route, close to the messages-only arm (28/48). Separating authority integrity (was this transition permitted) from decision integrity (did the permitted choice serve the principal) is the right framing, and the paper is careful to claim only channel integrity. Reporting that attacked joint reward went down under verified execution (57.5 to 46.9, the lowest of the adversarial arms), and disclosing protocol and governance deviations, is good practice.
To strengthen.
- Ground the "verified mandate" in real authorization mechanisms. As described, the mandate is an experimenter-set entry in a simulator-held registry, sender identity comes from simulator state, and sender claims in the tool call are ignored. That is an ACL check behind a trusted reference monitor, so zero accepted unmandated instructions holds by construction once identity and registry are trusted (the paper itself calls H-X0 a deterministic check and names those trust assumptions). The security-relevant questions start where those assumptions weaken: mandates carried as signed delegation or capability tokens (scoped, audience-bound, expiring, attenuable), OAuth-style scopes at an MCP or tool gateway, token theft and replay, confused-deputy chains, revocation races and nested delegation. Even one arm in which the adversary can try to obtain, replay or over-scope a bearer mandate would turn a design property into a tested result. The paper already lists contested mandates, compromised registries and a comparison with a conventional authorization stack of equal strength as follow-ups; engaging the authenticated-delegation and capability literature directly would place the contribution.
- Treat conversational bypass as the main result and pin down the mechanism. The route label cannot tell whether the honest agent was persuaded or would have chosen the target anyway; a no-adversary baseline for target selection on the same seeds, plus message ablation or counterfactual replay, would give a causal estimate. Relatedly, 42 of the 44 interception-route redirections in A_X only confirmed a task the recipient had already chosen and just two were genuine overrides, so it is not obvious why blocking the instruction lowers target selection by 16 trials; showing transcripts or allocation traces for the arms would help. Report the content of the S_XN notice, since its non-significant effect (25/48 vs 30/48) is hard to interpret without it.
- Exercise the benign path. H2 passes on average, but five of sixteen seeds fell below the margin and no benign agent issued a structured instruction, so false refusals, latency and legitimate-delegation cost were never tested. A benign workload with valid, invalid and revoked mandated instructions is needed before claiming utility is preserved.
- Tighten the accounting. State the unit for the delegation results (D_C and D_X are eight runs each, reported as 14/16 and 16/16 recipients); say how the remaining two of A_X's 46 redirections occurred, given 44 followed the interception route; and reconcile the "no parity violation" statement in Section 3.2 with the Appendix A note that the old whole-slot checker is false for the one replaced slot.
- Broaden beyond one 32B model and one Jira-style allocation task, and publish the repository and evidence bundle so others can check the cited hashes and the AI-assisted reproduction. Move framing material (Threshold 007, the separability thesis, "Effective Capability", which is only mapped to benign joint reward in Appendix C) out of the main text or define it there; it takes space without adding to the empirical claims, and the SC "Steganographic Collusion" series label does not match a study with no steganography.
Overall. A carefully run, tightly scoped experiment showing that structured-channel authorization works as designed but leaves most of the persuasion problem in place. The rigor is above typical sprint work. Impact is limited because the enforcement mechanism is an idealized registry checked against a synthetic no-permission baseline, so the strongest positive result follows from the setup; the residual-influence and welfare findings are the parts worth building on.
Read full reviewShow less
Cite this project
@misc{atanda2026who,
title = {{Who Controls Your Agent? Verified Mandates, Persistent Conversational Influence, and the Limits of Authority Enforcement}},
author = {Dele Atanda},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/who-controls-your-agent-verified-mandates-persistent-conversational-influence-and-the-limits-of-authority-enforcement-bkat}},
url = {https://apartresearch.com/sprints/projects/who-controls-your-agent-verified-mandates-persistent-conversational-influence-and-the-limits-of-authority-enforcement-bkat}
}More from AI Collusion Research Sprint
- View project: Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence
How can we distinguish genuine coordination between AI agents from coincidental behavioral alignment? We introduce a randomized canary audit that tests whether private information propagates through shared tool state, …
- View project: (1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
(1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
Bid Brother
Can an overseer find two secretly coordinating AI agents just by watching what they do? We tested pairwise net influence (Bonjour et al., 2022) on LLM bidders in a repeated procurement auction, with instructed cartels …
- View project: AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
Fugue
AI agents are starting to negotiate and bid for competing companies. We follow cartels among them from the first proposal to the price the buyer pays, in a sealed-bid procurement market where twelve LLM agents act as …