Talk Is Not Collusion: Do published collusion detectors catch LLM agents that collude on their own?

Collusion detectors for AI agents are usually tested on agents that researchers programmed to hide a secret signal. We tested whether a published detector suite, Audit the Whisper, catches language-model agents that collude on their own. In a simulated market where sellers set prices and can message each other, we ran 10 matched pairs of sessions. Groups of five GPT-4.1 sellers kept prices above the competitive level in all 10 sessions, while groups of three GPT-4.1 and two Claude Haiku sellers priced near cost in all 10 (exact paired test, p = 0.002). We scored detectors only on the GPT-4.1 sellers, so writing style could not separate the markets. Neither the suite’s detectors nor an AI judge using the market’s own monitoring prompt could reliably tell them apart. The judge did best, with a ROC AUC of 0.77 versus 1.00 for prices, because sellers in both markets used the same collusive language. A price rule using only the first five rounds flagged every colluding market with no false alarms. We also found that the suite’s three comparison methods, reported with perfect scores, are given the colluders’ identities as input. Monitors should therefore check outcomes as well as messages. All logs, code, and results are public, and one command recomputes the headline numbers.