What a Monitor Sees Matters More Than Its Size
Aleksei Kislov
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
AI agents increasingly act on people’s behalf, creating opportunities for secret coordination at others’ expense. LLM monitors offer a potential defence, but what evidence should they receive? I test whether combining agents’ messages and actions improves collusion detection, and whether a more capable monitor performs better with the same inputs. In a four-player El Farol game, two agents were secretly instructed to maximise a shared score while concealing their team. For each player model (Claude Haiku 5.5 and Opus 5.5) I ran 30 secret-team and 30 no-team games. Haiku and Opus monitors scored each game from its messages, actions, both, said-vs-did statistics or private plans. Haiku teams colluded profitably; Opus teams kept a fair rotation and gained nothing. On Haiku games, the Haiku monitor performed best with actions alone (AUROC 0.75). Messages alone were at chance (0.46), and adding them to actions reduced detection (0.60). The larger Opus monitor performed at chance across public views, while scoring cost 16 times as much. Greater reasoning effort increased the Haiku monitor’s suspicion without improving discrimination. With private plans, both monitors perfectly distinguished team from no-team games. What a monitor sees mattered more than its size or reasoning effort.
Reviews
Clear question and sound methods. The Haiku monitor beating Opus on actions is interesting, but two models from one family can’t settle size versus input. Opus games contain almost no collusion, and the prompt gives the monitor the base rate. Remove the base rate and add a third monitor.
Cite this project
@misc{kislov2026monitor,
title = {{What a Monitor Sees Matters More Than Its Size}},
author = {Aleksei Kislov},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/what-a-monitor-sees-matters-more-than-its-size-domy}},
url = {https://apartresearch.com/sprints/projects/what-a-monitor-sees-matters-more-than-its-size-domy}
}More from AI Collusion Research Sprint
- 1st placeView project: Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Collusion audits ask whether the actions of different agents depend on each other. Agents built on one model act alike without communicating, so these audits accuse honest fleets. Among four honest Claude Haiku agents …
- 2nd placeView project: Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Stego-probe
AI agents can collude without saying anything suspicious, just by choosing which true facts to mention. We built a hiring task where Qwen3-8B invents its own covert signals that get past an LLM overseer, then asked …
- 3rd placeView project: Do collusion probes generalise? Prompt confounds in NARCBench
Do collusion probes generalise? Prompt confounds in NARCBench
I investigated HELIOS, an activation-probe for detecting collusion. I found some apparent flaws, and analysed the (acknowledged) unreliable generalisation of the approach.