AI Collusion Sprint - Quiet Suppression

Strengths. The setting is well chosen: a grievance pipeline puts the cost of suppression on a complainant who never sees the handoff, which is closer to real enterprise agent deployments (support triage, claims, whistleblower intake) than most market-pricing collusion testbeds. The condition design is clean, with C3 (talk, no incentive) and C5 (incentive, no talk) as hard negatives around the C4 positive control, and the gap between stated severity and the escalation action (mean severity barely moves while escalation falls from 94% to 30%) is a useful say-versus-do result. The strongest contribution is "precedent capture": separating unsafe precedents, board endorsement and verdict polarity across six matched arms shows that the endorsement itself, not exposure to the cases, drives the watchdog failure, and the one-sentence policy-priority control is a concrete, testable mitigation. Breadth across seventeen watchdogs, the harder under-rated set showing the easy set was near ceiling, the one-line rule baseline, the sentence ablation and the seed-1 repeat of the two contrasting open-weight models are all well judged. The self-disclosure is unusually thorough: prior-work split by date and person, the input-hash defect in Appendix H, Opus refusals inflating its zero false-flag count, in-sample thresholds, and unvalidated labels. The summary tables in the repository match the paper (I recomputed the pooled harder-set counts, the 6/1,440 and 7/855 false-flag totals, and the interval criterion behind the abstract's eight-of-thirteen and seven-of-eight claims).

To strengthen.

1. Put the read-level intervals in the PDF. The abstract's eight-of-thirteen and seven-of-eight counts rest on a non-overlap criterion whose intervals appear only in the repository tables, and the PDF never names the eight. Under that criterion MiniMax (85% under endorsement) does not count as a drop, while Section 4.3 says endorsement lowered detection "in all four families". A single table listing each watchdog's A/C/F flag rate with interval, and which ones meet the criterion, would make the headline claim checkable from the report alone.
2. Release text-free per-read observations (case ID, arm, draw, model, flag, score, recomputed input hash). The repo cannot regenerate any table, raw logs sit on a team Drive or "on request", and Appendix H shows the per-row hashes for 228 of 324 easy-set rows do not match transmitted text. Publishing the sidecar mapping you describe would close both gaps without exposing case text.
3. Add a held-out evaluation for the harder set. With 13 suppression items on six records, labels taken from the same honest pipeline, an in-sample 7-of-9 threshold and in-sample ROC thresholds, the 38% to 15% to 27% result is suggestive rather than established; the Qwen baseline moving from 7 to 13 of 26 between seeds shows how wide these cells are. A frozen set of new complaints with human-checked severity, scored by complaint rather than by read, is the right next step, as your Future Work says.
4. Mark in the main text which numbers come from pre-sprint runs. Appendix B says the cached MiniMax trajectories and the first MiniMax pressure, paraphrase and watchdog-position pilots are prior work, but Sections 4.1–4.3 present those figures alongside sprint results without distinction.
5. Tighten condition labelling. The no-aim baseline in Figure 1 and Table 2 is the C3 notes pipeline, while Table 1 calls C2 the baseline; C1 and C6 appear only in appendices; "nine-stage" in 4.2 versus seven roles in 3.2. Also consider separating the endorsement effect from a generic conflicting-instruction effect, for example an arm where unsafe precedents are endorsed by a low-authority source.

Overall. A well-designed, carefully hedged study with a practically relevant finding: calibration examples endorsed by an authority can override a watchdog's written policy, and a one-line priority statement recovers detection for some models but not others. The main limits are statistical depth (few seeds, small harder set, no human labels) and reproducibility from the public package. The defensive implication, that calibration material for AI reviewers needs independent review, is useful to practitioners now.