Does splitting an LLM watcher's job improve its collusion detection?
Marion Nowicki · Team AI & Becoming
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
In Paper 1, a patrol of scheduled price cuts lowered collusion between two LLM sellers with equal costs and no shared board. Here we ask who should watch the market, and whether splitting the watcher's job helps. In 218 markets of three LLM sellers (DeepSeek v4.1 flash or Claude Haiku 5.5), one with a lower cost, with and without a shared board and the patrol, we first find that the same patrol does not adapt to this market: it took most patrol markets below the competitive price, including Haiku markets whose prices were already at the competitive level. A watcher is needed to say when to cut. We then compare LLM watchers on two questions, whether the sellers wrote an agreement and whether prices followed. A reader following a watcher trades one mistake for the other: it believes cheap talk, or, without the sellers' words, misses their agreements. Splitting the reading job again, with two readers after the watcher, one for the board and one for the prices (given each seller's competitive and joint-profit prices), sorted the most markets into the right case on both seller models (75 of 87 DeepSeek and 80 of 100 Haiku markets; the best watcher with one reader, 53 and 60) and kept its lead at temperature 1.
Reviews
Crossing whether sellers wrote an agreement with whether prices followed, for me, is a good way to make cheap talk and parallel moves visible. One result is a useful warning: a single watcher believes cheap talk (22 of 25 markets) while still reporting most real agreements. Pre-registration, hashed prompts, blinded labels, reported failed predictions and a full data release are good practice.
The main problem is that the design doesn't isolate the effect of splitting. Only the split watcher is given each seller's competitive and joint-profit prices. It also uses a different model and makes an extra call. With that price scale, its price half simply re-derives the label. Each model also makes only one half hard, with every DeepSeek board holds an agreement, and no Haiku market reached high prices.
Cite this project
@misc{nowicki2026splitting,
title = {{Does splitting an LLM watcher's job improve its collusion detection?}},
author = {Marion Nowicki},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/does-splitting-an-llm-watchers-job-improve-its-collusion-detection-7kfh}},
url = {https://apartresearch.com/sprints/projects/does-splitting-an-llm-watchers-job-improve-its-collusion-detection-7kfh}
}More from AI Collusion Research Sprint
- 1st placeView project: Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Monoculture Looks Like Collusion: A Replay Test for Hidden Information Flow Between LLM Agents
Collusion audits ask whether the actions of different agents depend on each other. Agents built on one model act alike without communicating, so these audits accuse honest fleets. Among four honest Claude Haiku agents …
- 2nd placeView project: Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Looking for a Hidden Agenda: Can a Model’s Activations Reveal Steganography that its Text Hides?
Stego-probe
AI agents can collude without saying anything suspicious, just by choosing which true facts to mention. We built a hiring task where Qwen3-8B invents its own covert signals that get past an LLM overseer, then asked …
- 3rd placeView project: Do collusion probes generalise? Prompt confounds in NARCBench
Do collusion probes generalise? Prompt confounds in NARCBench
I investigated HELIOS, an activation-probe for detecting collusion. I found some apparent flaws, and analysed the (acknowledged) unreliable generalisation of the approach.