Verify, Don’t Decode: When Trusted Agents Disrupt LLM Collusion in Software Engineering and Double-Auction Markets

The report tests whether trusted agents placed inside a multi-agent system can reduce the harm from collusion without first decoding how the colluders communicate. It does this in two settings: a double-auction market and a software team working on a real FastAPI change. The work is pre-registered, reports null results as nulls, and records what failed in the pilots and in the first chain-of-monitors run. The lessons in the discussion are useful for anyone designing layered oversight. A missing verdict from an untrusted monitor should count as a failed audit rather than consent; honeypots must match real cases in every field the monitor can see; and a verifier’s “pass” only covers the tests it runs.

The market results need more context. The report itself shows that the entrant’s gains came mostly from its own trades, while incumbents kept pricing near the monopoly level. Adding a low-priced seller to a market with five sellers and four buyers will raise buyer surplus whether or not the cartel is disrupted. Removing sellers will raise prices in the same market for the same reason. Both findings are expected given supply and demand, so they say more about this market’s size than about the value of trusted agents. The report should also explain who would act as the entrant in practice and how it would sustain selling near cost.

The steganographic condition does not appear to test steganography. The A5 sellers followed their price code in only 3 of 10 coded hours, and only 2 of 36 review windows carried the partner code, yet A5 still reached a collusion index of 0.98. The high prices therefore came from the instruction to collude rather than from a hidden channel. Conclusions that rest on the difference between open and hidden coordination, such as the finding that publishing the competitive price dissolved the open cartel but not the hidden one, should be stated more tentatively.

In the software setting, the first study could not test disruption because only 6 of 49 teams wrote correct code, which the report acknowledges. In the second study, the abstract says the verifier “tripled correct releases,” but the comparison is 6/10 against 2/10 with p = 0.17, so the effect is not established. The collaborator’s T1 runs point the other way: with the verifier present, teams wrote correct code less often (4/10 against 10/10, p = 0.01). This result deserves more attention than a single sentence. In the chain-of-monitors study, the scheming reviewer gave no verdict in 30 of 36 reviews because its reasoning ran past the reply limit. This is partly an artifact of the token limit, not only a strategic choice, and the fail-closed fix also distrusted the honest reviewer in 4 of 6 episodes.