Talk, Actions, and Planted Approval Pacts: An Exploratory Audit

We explored whether combining agent messages with approval actions helps distinguish coordinated false approval from other behavior. Two language-model reviewers assessed an inert change against an executable numeric requirement. Across 60 episodes and six assigned conditions, we compared three fixed detectors: a keyword-based talk detector, an action detector, and their conjunction.

The frozen analysis failed its positive-control threshold: four observed positives against a required five. Subsequent judge retries, added after inspecting that count, support exploratory analysis only. Under the final model-judge labels, the hybrid detector flagged four explicit-agreement episodes and no judge-labeled negatives, while missing three concealed-agreement positives. The talk detector also flagged all ten benign negations containing its target phrase.

Reviewer disagreement exposed a central measurement problem: bilateral approval of a failing change does not, by itself, settle whether an agreement occurred. Results depended on the definition of agreement and the evidence available to reviewers. We disclose protocol deviations, reviewer provenance gaps, and the distinction between scripted controls and live model outputs. This pilot demonstrates limitations of simple monitoring rules; it does not establish spontaneous collusion or deployment accuracy.