Did They Collude or Just Agree? What Counts as Evidence of Collusion Among AI Reviewers
Sam Matuba · Team Asimov
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Two reviewers vouch for a claim, and the claim turns out to be false. Did they collude, or only agree? A shared source, copying, a persuasive proposer, and a shared incentive all leave the same records. I built a small review game to find out what can tell these causes apart. A proposer selects evidence, two reviewers endorse or reject, a decider approves, and an auditor reads the record later. Eight settings cover an honest baseline, four innocent causes, a chance to collude, talk without action, and a scripted pact. I fixed the hypotheses and tests before the data and ran one language model as the confirmatory test. Six more models followed with the same prompts, 288 plays each, as an exploratory check. Three results held for all seven. The private channel never made the decider approve more false claims; in two models it made fewer. Provenance data did not help the auditors, because they almost never accused an innocent record. A keyword auditor found my scripted pact and missed the pacts the models wrote in their own words. In the three models classified by hand, words did not predict actions. Of 108 classified records, one showed a pact that changed an action. Only an auditor that could compare the record with the same case played without the channel caught it. A real auditor has no such second record. Harm alone and messages alone are not evidence of collusion. Convincing evidence must link a plan to a change in action.
Reviews
No public critique yet.
Cite this project
@misc{matuba2026did,
title = {{Did They Collude or Just Agree? What Counts as Evidence of Collusion Among AI Reviewers}},
author = {Sam Matuba},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/did-they-collude-or-just-agree-what-counts-as-evidence-of-collusion-among-ai-reviewers-0f96}},
url = {https://apartresearch.com/sprints/projects/did-they-collude-or-just-agree-what-counts-as-evidence-of-collusion-among-ai-reviewers-0f96}
}More from AI Collusion Research Sprint
- View project: Is It a Cartel? Separating Enforced from Spurious Collusion in LLM Pricing Agents
Is It a Cartel? Separating Enforced from Spurious Collusion in LLM Pricing Agents
When LLM pricing agents keep prices above the competitive level, are they running a cartel, or are they simply stuck? Only the first is collusion, and the two call for opposite remedies. We introduce a black-box causal …
- View project: Every Permission Checked: Source-Linked Evidence for Auditing Finance Agents
Every Permission Checked: Source-Linked Evidence for Auditing Finance Agents
When multiple AI agents contribute to a policy violation, the same outcome can reflect cooperation, deception of a helper, or an independent mistake. We study how evidence presentation affects this distinction in a …
- View project: Swapping the Model Does Not Reset the Price: What LLM Pricing Agents Inherit After a Model Replacement
Swapping the Model Does Not Reset the Price: What LLM Pricing Agents Inherit After a Model Replacement
Emerald
We test what happens when one of two LLM pricing agents has its model replaced, the open problem posed in the sprint's question 1.2. Two gpt-oss-120b agents priced for 150 rounds; we then swapped one agent's model to …