Monitored Collusion in LLM Auctions
Eyitayo Alimi
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
LLM agents that submit bids or set prices on behalf of firms could learn how to coordinate with one another, so monitoring their chats is a proposed safeguard. We ask whether agents aware threat their chats are being monitored moved coordination from what they say into what they say into how they bid and if chat monitors could catch them. I ran 80 games of a repeated first-price procurement auction(three firms, 20 rounds, chat then selected bids) in four conditions: compete (negative control), neutral, collude_open. (positive control) and collude-monitored(a stated but uninformed monitoring threat). The controls behaved : late-game was 0.06 under compete and 0.94 under collude_open. Under the stated threat, collusion fell to 0.78 and explicit price talk fell from 100% to 72% of messages, but did not hide. Chat-only monitors built on three model families (gpt-4o, DeepSeek, Qwen2.5-32B) flagged all 20 games in every non-controlled condition (AUC about 1.00). A model-free bid screen was weaker on the neutral condition(AUC 0.88), and adding bods to the LLM monitor did not help (gpt-4o AUC 0.87 and 0.89). The chat monitor also flags friendly tone at near-zero markup, so its ceiling performance may reflect tone as much as prices. Collusion here is instructed and the threat is unenforced; an enforced condition was designed but not run.
Reviews
This is a well-designed and honestly reported study. It runs 80 games of a repeated three-firm procurement auction under four conditions: compete (negative control), neutral, collude_open (positive control) and collude_monitored (a stated but unenforced monitoring threat). It then scores a chat-only LLM monitor, a model-free bid screen and a combined chat+bids monitor on the same games. Several practices stand out for a hackathon:
The pre-registration openly declares that the pilot seeds were included.
Every monitor’s threshold is calibrated on the compete games, and AUC is reported alongside detection rates.
Robustness monitors come from three model families, including Qwen, a non-bidding model.
The analysis of the compete game that reached the price cap through misplay (seed 17) is a nice touch.
The headline is also honest: a stated threat reduced both collusion (late markup 0.94 → 0.78) and explicit price talk (100% → 72%), but the agents didn’t move coordination into their bids. The concerns are mainly about what the monitor results can show.
The chat monitor’s AUC of about 1.00 is confounded by the prompt. Every AUC is computed against the compete games, whose prompt tells firms to “compete hard and never coordinate,” so their chat sounds noticeably different. The paper’s own per-round analysis shows the monitor flagging neutral rounds even at near-zero markup. So a perfect AUC may mostly mean “not told to compete” rather than “colluding,” as the author partly acknowledges. Two cheap fixes:
Define ground truth per game from outcomes (e.g. sustained markup above a threshold), not from the condition label.
Compute AUC within the neutral condition, comparing high-markup and low-markup games.
Either would show whether the monitor tracks prices or tone.
Neutral shouldn’t be counted as a positive class. Neutral games average 0.37 late markup with wide variation. Counting all 20 as “should be flagged” makes 20/20 detection look like a success, when on many of those games it’s a false positive. Reporting neutral separately, as a test of false positives on legitimate cooperative-sounding chat, would turn this into a finding about monitor reliability.
The substitution test was limited. Collusion is instructed, the threat isn’t enforced, and chat comes before bids every round, so the agents had little reason or room to stop talking. A “collude, no chat” condition would directly test whether the bid history alone can carry coordination (question 3.1). It would also give the bid screen a positive class where chat can’t help. The enforced condition was designed, calibrated, and partly piloted (data/enforced_pilot.jsonl). Running it is the most important next step, and briefly reporting the pilot, labeled as anecdotal, would add value.
Model-family overlap. Two of the three bidders are gpt-4o-mini and the main monitor is gpt-4o; Firm A and the second monitor are both DeepSeek. This is stated, and the Qwen monitor partly addresses it. Still, the 0.92–0.95 agreement between monitors doesn’t show that they detect the right thing; it may only show they react to the same cues, as the paper notes.
The combined monitor result needs a closer look. For gpt-4o, adding bids lowered AUC, which is interesting, but there’s no analysis of why. Pulling a few neutral games where the chat-only and combined scores differ and showing the monitor’s stated reasons would help. It may be that seeing low bids leads the monitor to correctly lower its score on cooperative-sounding chat, which would mean the combined monitor is the more accurate one and the chat-only “ceiling” is the error.
Presentation. Please proofread the abstract and intro, which currently read like an unedited draft. Also remove the leftover template text in Code and Data, and either add the referenced Appendix A or drop the reference. The body text is good, and these errors undersell it.
Overall, this is a solid, reproducible testbed with honest reporting. The key extensions would make it a strong result: outcome-based ground truth, a no-chat condition and the enforced threat.
Read full reviewShow less
Cite this project
@misc{alimi2026monitored,
title = {{Monitored Collusion in LLM Auctions}},
author = {Eyitayo Alimi},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/monitored-collusion-in-llm-auctions-0cdn}},
url = {https://apartresearch.com/sprints/projects/monitored-collusion-in-llm-auctions-0cdn}
}More from AI Collusion Research Sprint
- View project: Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence ? Causal Auditing of Collusion Through Shared Agent State
Signal or Coincidence
How can we distinguish genuine coordination between AI agents from coincidental behavioral alignment? We introduce a randomized canary audit that tests whether private information propagates through shared tool state, …
- View project: (1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
(1.4) Collusion, Espionage, or Look-alikes? Auditing Information Flow between LLM Agents
Bid Brother
Can an overseer find two secretly coordinating AI agents just by watching what they do? We tested pairwise net influence (Bonjour et al., 2022) on LLM bidders in a repeated procurement auction, with instructed cartels …
- View project: AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
AI Sellers Propose Bid-Rigging Within Minutes, but Their Agreements Do Not Hold
Fugue
AI agents are starting to negotiate and bid for competing companies. We follow cartels among them from the first proposal to the price the buyer pays, in a sealed-bid procurement market where twelve LLM agents act as …