Detecting Collusion with Pairwise Net Influence: A Baseline Comparison in Toy Multi-Agent Games

This project implements the pairwise net influence detector from Bonjour et al. (2022) and compares it with a simple baseline the original paper lacks. The work is careful and honest. The code is short, readable, unit-tested and fully reproducible, and it matches the methods description. The confidence intervals, the disagreement analysis and the “What did not work” section are all good practice. The conclusion, that there is no evidence net influence beats Cramér’s V here, is stated with appropriate restraint. The main concern is whether this setup could have shown a difference at all.

The baseline is almost the same statistic as the detector’s own input. For a 3×3 table, mutual information and Cramér’s V are nearly monotone transforms of each other, because MI is roughly proportional to χ²/n when dependence is weak. I checked this with the repo’s own simulators. Across 3,600 games covering all four scenarios, raw MI and Cramér’s V chose the same top pair in 3,590 (99.7%). So the real comparison is net influence with the outside-influence subtraction versus without it, not “information-theoretic vs correlation.” Framing it that way would make the finding sharper.

The decoys can’t be told apart from collusion, so no dependence-based detector could separate them. In every scenario, each decoy link is a direct copy or beat rule with the same form as the planted collusion. The separate-decoy pair (0,1) behaves exactly like a colluding pair and is “innocent” only by label. When the decoy is stronger than the real pair, every method that ranks by dependence strength should pick the decoy. The paper notes this follows from the design, but it means the headline curves mostly measure “p vs q.”

The case net influence was designed for isn’t measured. The subtraction exists to stop a third agent from creating apparent collusion. In the hub scenario, agents 1 and 2 both copy agent 0, so the pair (1,2) looks dependent without any direct link between them. That indirect link is always weaker than the direct links behind it, so it is almost never the top-ranked pair. The question that matters is whether it gets falsely flagged, and ranking only the top pair hides that. The ranking rule also drops the threshold, which is the paper’s actual decision mechanism. Without it, specificity can’t be assessed at all. A further side effect: when a colluder also has a strong innocent link (hub/chain), the subtraction penalizes the true pair, which may explain some of the “both wrong” cases.

A suggested redesign. Add no-collusion and confounder-only conditions. Apply the paper’s threshold α and report false-positive and true-positive rates (ROC curves) for each method. Make sure at least one condition has an indirect, confounded link stronger than the real collusion. A setting where the common driver is partly unobserved would make the comparison meaningful.

Minor issues.

The intro describes Fish et al. (2024) as showing “sub-competitive” pricing; their finding was supracompetitive pricing.

Raw LaTeX (\(\gamma(i;j)\)) appears in the intro.

The reproducibility paragraph still says “(add the link).”

“Collab” should be “Colab,” and the repo name has a typo (“detetction”).

Seeds shared across p values make the points on each curve correlated, which the paper rightly notes.

Agent 3 “beats agent 2’s move” in the same round, which assumes agent 3 can see that move. That’s fine for a toy model but worth stating, since the game is described as simultaneous.

Overall, this is a solid, well-documented negative result. With a false-positive test and a confounder scenario built for the purpose, it would directly answer the question it poses.