Swarm-Referee: Can One Fixed-Prompt Referee Find Planted Collusion in Multi-Agent Dinner Logs?Ω
Ashish Raina · Team Swarm-Referee
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Does a monitor that reads both what agents say and what they do flag collusion more than a monitor that reads only one of them? We tested this in a pilot of 16 games of the Diners game (Shi et al.) with five LLM agents and five rounds. We planted collusion by instruction in 6 games: two agents received a pre-dinner pact to promise "cheap" and order "expensive". Four decoy classes used the same payoffs: independent cheats, a table-wide cheap pact, collusive talk with no matching orders, and no instruction. One frozen referee (deepseek-v4-pro, one fixed prompt, temperature 0) read each game in five views, from messages only to a ledger with marks for stated-versus-actual orders. Two rule baselines served as comparison. One author labelled all 16 games by hand. The referee flagged the planted pair in 6 of 6 games on the ledger view (95% CI 54.1–100%). It also flagged 5 of the 10 games the author labelled not collusion (50%). We found no evidence of detection from messages alone or from actions alone in 16 runs (one-sided 95% upper bound on recall 39.3%). Our evidence check found no message cue that separates an agreed pair from independent cheats, so the pair-versus-independents question is unanswerable from these logs. All collusion here is planted.
Reviews
No public critique yet.
Cite this project
@misc{raina2026swarmreferee,
title = {{Swarm-Referee: Can One Fixed-Prompt Referee Find Planted Collusion in Multi-Agent Dinner Logs?Ω}},
author = {Ashish Raina},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/swarmreferee-can-one-fixedprompt-referee-find-planted-collusion-in-multiagent-dinner-logs-ifh3}},
url = {https://apartresearch.com/sprints/projects/swarmreferee-can-one-fixedprompt-referee-find-planted-collusion-in-multiagent-dinner-logs-ifh3}
}More from AI Collusion Research Sprint
- View project: Collusion, Competion and Collapse
Collusion, Competion and Collapse
Atelier Alex Cline
Do AI traders collude when nobody tells them to? We built agora, a six-market benchmark with exact competitive and collusive benchmarks, and ran seven open-weight 2-4B models in 590 completed games under five …
- View project: Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Rogue Agents in Jail 🕵️🕵️♀️👮♂️
In order to advance AI safety research into shaping public policy, the development of technical controls needs to have a clear purpose, evidence, and limitations that are understandable for the people responsible for …
- View project: Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Gamma Ray
We test whether the information-theoretic collusion detector of Bonjour et al, validated on scripted agents, works on LLM agents. Gemma, Mistral and Qwen played 1000 rounds of three-player Rock-Paper-Scissors in three …