Collusion Lives in Memory: Localising and Stress-Testing Tacit Coordination Between LLM Agents

Quite creative. This asked a question with real value for defenders. Not just whether agents collude, but where the coordination is stored.

The warning that single-replay audits mislead (5 of 9 flagged pathways were false positives) is useful on its own. The transplant experiment is interesting, with sanitised notes still passed the market split to fresh firms in 6 of 6 games, against 0 of 6 with non-colluders' notes.

Needs more in-depth experimentations and rescoping:

• Narrow the conclusion. The headline rests on 4 of 5 split games, and in the monitored games the robust carrier was the board rather than memory.
• Validate the sanitiser. It kept directives such as holding one good at zero. Check it, or add a hand-written-note control, before concluding that content filtering fails.
• Add a baseline. Run a symmetric-cost baseline with no channels.
• Compute AUROCs within each arm, and use more replays per pathway with permutation tests.
• Fix Fig. 2's axis labels and explain the rankers.

Repeating each replay instead of trusting a single one was a really good call, and the finding that most first-pass flags didn't survive is useful for anyone auditing agents. The transplant result surprised me. You describe Firm B completing the split through market feedback, and I wasn't sure how much the note carries beyond the "produce only Y". A scripted firm that just produces Y could help tell those apart. I'd also be curious how a monitoring notice without the word 'collusion' may change the reasoning traces.

This is a strong sprint paper, and the finding sticks with you. Tacit collusion between these LLM firms lives in private memory. Clean it up, strip out every mention of the rival and of dividing the market, and it still carries over. Fresh firms handed those sanitised notes split the market with new rivals. That is the result I kept coming back to.

The audit method is the other half of why this paper works. An outside recorder logs everything, the provenance graph traces where each decision came from, and then the authors replay each game again and again with one pathway masked, against matched null replays. It is a clear step past deleting a whole channel at once, or asking an LLM judge to read a transcript and guess. The paper proves the point on its own data. A single-replay audit flagged five wrong pathways and missed two real ones. One replay just is not enough with stochastic agents, and the authors show that rather than only saying it.

The transplant experiment is the standout. Six out of six splits from sanitised colluder notes, against zero out of six from non-colluders' notes. The controls are the right ones, and the implication is hard to ignore for anyone running persistent memory in deployed agents. If a cleaned-up note can pass a collusive rule to a firm that was never in the original game, then memory is not just storage. It is a way the behaviour spreads.

A few things would make this stronger. First, the samples are small. Five audited games, six per transplant arm, and a single model. More games and at least one more model that actually colludes would go a long way. Second, the sanitisation result still rests on single replays, which is exactly the procedure the paper shows to be unreliable. It needs to be re-run under the same repeated-replay rule as the main audit. Third, seed 12 deserves more than a sentence. In that game no pathway held up under repetition, and the split was held by market feedback alone. That is a real boundary on the claim that memory carries collusion, and I would like to see the authors sit with it. Fourth, the ranker comparison and the behavioural monitor signal are exploratory, and they should stay labelled that way. They largely are. Pre-registering them and testing on fresh games is the natural next step.

The writing is clear, and the limitations section is unusually honest. I trusted the strong claims more because the authors were straight about what they had not shown.