GRAFT++: Reducing Collusive Behavior in Agents via Localized Training and J-space Monitoring

Large language model (LLM) agents now set prices on behalf of companies, and agents that can communicate can agree to raise prices at the expense of customers. A natural safeguard is a monitor that reads the messages of the agents and penalizes coordination, but it is not known whether agents that learn about such a monitor stop colluding or only stop discussing it. To study this question, we introduce the Hawthorne Market, a novel simulated market in which LLM agents act as competing sellers, set prices in repeated rounds and exchange private messages. Unlike earlier collusion benchmarks, which announce oversight from the start or change several conditions at once, the Hawthorne Market changes a single sentence of the prompt at a time, so that the effect of being watched can be isolated, and it records four signals behind every decision: the price, the message, the private reasoning and the internal activations of the model. A further novel element is a control group of sellers whose goal is units sold; these sellers cannot benefit from collusion and therefore show how agents react to monitoring when they have nothing to hide. We find that Qwen3-14B sellers whose only goal is profit form cartels without any instruction to cooperate. When told that a compliance team reviews their messages, they lower their prices but continue to coordinate in language that sounds compliant, and the control sellers change their messages in the same way. In a second setting, a double auction in which we continue already-formed cartels from saved states, we found that fines reduced the coordination messages of GPT-4.1 sellers from 100% to 7% while prices barely changed. A decline in flagged messages is therefore not reliable evidence that collusion has stopped. Internal activations remain informative, however: a linear probe detects hidden collusion more accurately than the message monitor, a trained message classifier and the price. Building on this result, we propose GRAFT++, a novel extension of gradient routing that uses the activation probe instead of a text monitor to decide which training examples update a set of weights that is deleted after training; to our knowledge, it is the first application of gradient routing to collusion among agents. On training data whose collusion the text monitor cannot see, GRAFT++ reduces collusion by about 60% relative to text-based routing (GRAFT), a significant difference that is smaller than the margin we pre-registered. These results suggest that platforms should not rely on message monitoring alone, and that signals from inside the model offer a practical basis for defenses during training.