Skip to content
Apart Research
Apart Research
Contact us
Sprint projectOct 26, 2026Canada

Talked or Told? Testing Information-Theoretic Collusion Detectors on LLM Dealers

yosra kazemi, Melanie Bui, My Chiffon Nguyen, Karan Verma, Linh Le, Hong Kiat Tan, David Williams-King · Team Lida Safety

Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Talked or Told? Testing Information-Theoretic Collusion Detectors on LLM Dealers

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

Can information-theoretic statistics localize LLM dealer pairs given private communication or coordination instructions, using only their observed actions? We test mutual information, its conditional form (CMI) and calibrated net influence in a three-dealer procurement auction with independent private costs, across 96 markets with Gemini 2.5 Flash and gpt-oss-20b. An assigned pair receives a channel, an instruction, or both; a shared-signal control supplies common information. CMI ranks the talking pair first in 8 of 9 Gemini and 9 of 9 gpt-oss markets, but the instruction-only pair in 5 of 18 and 1 of 9. Instructions change quoting, detected by price-level (Gemini) and own-cost (gpt-oss) screens. Localization and harm diverge: talking pairs lose profit without a detectable price increase, while weakly localized instruction-only Gemini pairs raise prices. Adding a channel makes instructed pairs localizable, yet lowers prices with Gemini and raises them with gpt-oss. Action dependence alone does not establish harmful collusion.

Reviews

Judging this Sprint?

Review “Talked or Told? Testing Information-Theoretic Collusion Detectors on LLM Dealers”

Your public critique appears on the project page without your name. Reviews of strong projects are shared with funders, as you choose below.

Not shown on the project page.

Shown on the project page, without your name.

Only the Apart team reads this, and funders if you agree below.

Decline to score

Only when the quality of the writing makes the work impossible to follow. Say why in your private critique; the project page shows only that a judge declined, not why.

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.
Sharing your name *

When we share a strong project with funders, your name, scores and public critique go with it, confidentially; this choice is only about whether your name is public. If you think this project deserves funding, please also endorse it: that carries the most weight.

Share my private critique with funders *

See our Privacy Policy.

  1. Detectors can identify comms between agents without necessarily identifying harmful collusion, and these same information-theoretic collusion detectors can miss collusive behavior that lacks statistical dependence. This is a helpful contribution that clarifies the limitations of these detectors. However, broader validation is needed to establish how well findings generalize.

    Strength: Experimental design helps prove that statistical dependence != harmful collusion

    Weakness: Narrow market structure, 2 model configs, small samples.

    Recommendation: Test across more realistic market conditions, coordination strategies, and other behaviors to further establish detector reliability and usefulness.

Cite this project

@misc{kazemi2026talked,
  title = {{Talked or Told? Testing Information-Theoretic Collusion Detectors on LLM Dealers}},
  author = {yosra kazemi and Melanie Bui and My Chiffon Nguyen and Karan Verma and Linh Le and Hong Kiat Tan and David Williams-King},
  year = {2026},
  month = oct,
  note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/talked-or-told-testing-informationtheoretic-collusion-detectors-on-llm-dealers-1xpq}},
  url = {https://apartresearch.com/sprints/projects/talked-or-told-testing-informationtheoretic-collusion-detectors-on-llm-dealers-1xpq}
}

More from AI Collusion Research Sprint

All 392 entriesBrowse all projects

A Sprint is running now

AI x Epistemics Research Sprint · Nov 13 - 15, 2026