Cheating Is Not Collusion: Says-vs-Does Monitors and a Matched Control for LLM Collusion Detection
Bobokhon Abdurakhmanov · Team Bob's Team
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Monitors usually see what agents say and what they do. We ask whether combining the two detects collusion better than either alone (Sprint question 2.2). Building on LLM-Trust-Breaking, we seed a secret pact between two of four agents in a Diner's Dilemma and add a matched control: two agents who receive the same cheating instruction individually, with no partner. A frozen gpt-4o-mini monitor judged 80 pre-registered games under four views. A claims-vs-actions table scored best (64% accuracy) and separated pacts from honest play (AUC 0.95), but no public view separated pacts from lone cheaters (AUC 0.41 to 0.62). This held across fresh games, prompts and monitors; only the agents' private plans carried the signal (AUC 0.86). We then gave agents an optional private chat with no collusion instruction. Across 119 gpt-5-mini chat games, the author-checked collusion count was 2 clear plus 1 borderline, while an action-based label fired in 106 of 120 games and the monitor flagged 98% of them. When collusion paid, concealed pacts rose to 4 of 30 games, and only the claims-vs-actions table caught some (42% of only 4 pact games) with few false alarms (9%). Two top-5 frontier models showed no collusion. Says-vs-does monitoring detects cheating, and collusion only when colluders lie in public; we propose the matched lone-cheater control as a minimum evidence standard.
Reviews
No public critique yet.
Cite this project
@misc{abdurakhmanov2026cheating,
title = {{Cheating Is Not Collusion: Says-vs-Does Monitors and a Matched Control for LLM Collusion Detection}},
author = {Bobokhon Abdurakhmanov},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/cheating-is-not-collusion-saysvsdoes-monitors-and-a-matched-control-for-llm-collusion-detection-7muv}},
url = {https://apartresearch.com/sprints/projects/cheating-is-not-collusion-saysvsdoes-monitors-and-a-matched-control-for-llm-collusion-detection-7muv}
}More from AI Collusion Research Sprint
- View project: Collusion, Competion and Collapse
Collusion, Competion and Collapse
Atelier Alex Cline
Do AI traders collude when nobody tells them to? We built agora, a six-market benchmark with exact competitive and collusive benchmarks, and ran seven open-weight 2-4B models in 590 completed games under five …
- View project: Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Rogue Agents in Jail 🕵️🕵️♀️👮♂️
In order to advance AI safety research into shaping public policy, the development of technical controls needs to have a clear purpose, evidence, and limitations that are understandable for the people responsible for …
- View project: Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Gamma Ray
We test whether the information-theoretic collusion detector of Bonjour et al, validated on scripted agents, works on LLM agents. Gemma, Mistral and Qwen played 1000 rounds of three-player Rock-Paper-Scissors in three …