Do Monitors Add Up Across Agents? Tool Logs, Chat and a Two-Agent Overspend
Jason Gibbs · Team Weis
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Track 2.2 asks whether combining what agents say with what they do beats either alone. We test it on the smallest coalition that can break a rule: two treasury agents, each limited to 100 per transaction, under a team rule that together they may not send more than 100 to one payee without a co-signer. In an exploratory, post-hoc run (sealed before any model call; 150 episodes, Mistral Small and Llama 3.3 70B), a judge reading both tool logs without the chat caught 41/41 violations and 11/11 joint-only overspends with 0/59 false alarms. Each agent's log alone caught 1/11 (judge or rule; paired 10 vs 0, p = 0.00098), and a flag-any-payment rule false-alarmed 55/59 (p = 3 × 10⁻¹⁷). The judge left alone all 5 episodes where one agent legitimately paid 100 alone, so it summed across agents rather than asking whether anyone paid. A chat-only judge caught 15/41 and 1/11; adding chat to the logs gained nothing. Caveats: 11 joint-only events, two models, one judge model, and an easy honest condition. An earlier pre-registered run (781 episodes, eight models) gave logs 11/11 vs chat 4/11, but there any payment already broke a single-agent rule, so it could not separate summing from "did anyone pay?"; this run removes that confound.
Reviews
No public critique yet.
Cite this project
@misc{gibbs2026monitors,
title = {{Do Monitors Add Up Across Agents? Tool Logs, Chat and a Two-Agent Overspend}},
author = {Jason Gibbs},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/do-monitors-add-up-across-agents-tool-logs-chat-and-a-twoagent-overspend-yjka}},
url = {https://apartresearch.com/sprints/projects/do-monitors-add-up-across-agents-tool-logs-chat-and-a-twoagent-overspend-yjka}
}More from AI Collusion Research Sprint
- View project: Collusion, Competion and Collapse
Collusion, Competion and Collapse
Atelier Alex Cline
Do AI traders collude when nobody tells them to? We built agora, a six-market benchmark with exact competitive and collusive benchmarks, and ran seven open-weight 2-4B models in 590 completed games under five …
- View project: Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Rogue Agents in Jail 🕵️🕵️♀️👮♂️
In order to advance AI safety research into shaping public policy, the development of technical controls needs to have a clear purpose, evidence, and limitations that are understandable for the people responsible for …
- View project: Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Gamma Ray
We test whether the information-theoretic collusion detector of Bonjour et al, validated on scripted agents, works on LLM agents. Gemma, Mistral and Qwen played 1000 rounds of three-player Rock-Paper-Scissors in three …