Talk Is Not Collusion: Do published collusion detectors catch LLM agents that collude on their own?
Rahul Kumar · Team Shinobi
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Collusion detectors for AI agents are usually tested on agents that researchers programmed to hide a secret signal. We tested whether a published detector suite, Audit the Whisper, catches language-model agents that collude on their own. In a simulated market where sellers set prices and can message each other, we ran 10 matched pairs of sessions. Groups of five GPT-4.1 sellers kept prices above the competitive level in all 10 sessions, while groups of three GPT-4.1 and two Claude Haiku sellers priced near cost in all 10 (exact paired test, p = 0.002). We scored detectors only on the GPT-4.1 sellers, so writing style could not separate the markets. Neither the suite’s detectors nor an AI judge using the market’s own monitoring prompt could reliably tell them apart. The judge did best, with a ROC AUC of 0.77 versus 1.00 for prices, because sellers in both markets used the same collusive language. A price rule using only the first five rounds flagged every colluding market with no false alarms. We also found that the suite’s three comparison methods, reported with perfect scores, are given the colluders’ identities as input. Monitors should therefore check outcomes as well as messages. All logs, code, and results are public, and one command recomputes the headline numbers.
Reviews
No public critique yet.
Cite this project
@misc{kumar2026talk,
title = {{Talk Is Not Collusion: Do published collusion detectors catch LLM agents that collude on their own?}},
author = {Rahul Kumar},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/talk-is-not-collusion-do-published-collusion-detectors-catch-llm-agents-that-collude-on-their-own-0ax8}},
url = {https://apartresearch.com/sprints/projects/talk-is-not-collusion-do-published-collusion-detectors-catch-llm-agents-that-collude-on-their-own-0ax8}
}More from AI Collusion Research Sprint
- View project: Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence
How can we distinguish genuine coordination between AI agents from coincidental behavioral alignment? We introduce a randomized canary audit that tests whether private information propagates through shared tool state, …
- View project: (1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
(1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
Bid Brother
Can an overseer find two secretly coordinating AI agents just by watching what they do? We tested pairwise net influence (Bonjour et al., 2022) on LLM bidders in a repeated procurement auction, with instructed cartels …
- View project: AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
Fugue
AI agents are starting to negotiate and bid for competing companies. We follow cartels among them from the first proposal to the price the buyer pays, in a sealed-bid procurement market where twelve LLM agents act as …