Evaluation Before Regulation: How LLM Pricing Agents Come to Collude, and What Stops It

This is a strong paper that suggests that before trying to stop AI pricing agents from keeping prices high, first figure out why they are doing it. The main findings are clear and easy to follow. Hiding competitors’ prices did not help, and wiping the agents’ memory actually made prices go up, but simply showing each agent its best price stopped high prices in every game. These findings are the most useful part, so in my opinion they should be highlighted and shared upfront. The finding that agents’ written reasoning can be misleading is interesting, but it is concluded based on only a small number of games, so testing it more would make it more convincing. One obvious next step is to try the fixes in more realistic settings, because the best ones depend on knowing the right price, which real platforms may not know. It would also help to explain more simply how an agent following a price cut differs from a punishing one, since both can keep prices high. That could be another area for further exploration.

I like the idea of diagnosing why high prices persist before choosing an intervention. The forced-deviation experiments and comparisons across remedies provide more insight than simply testing whether a compliance instruction lowers prices.

I would be more cautious about the sharp distinction between following and enforcement. Matching a rival’s price cut can discourage future undercutting, even if the agent describes its motive as recovering sales rather than punishing its rival. Your observation that matching can make cutting unprofitable suggests that this distinction deserves more care. The results provide evidence against particular punishment patterns; the abstract’s broader conclusion about how LLM agents collude seems stronger than the tests establish.

The best-response intervention is interesting, but its success through deference raises another question. Would agents follow an inaccurate recommendation just as readily? Testing noisy or deliberately perturbed recommendations would help distinguish the value of better information from obedience to a platform-supplied number. It would also make the practical implications more convincing, since estimating the relevant demand and profit functions is itself a substantial problem in real markets.

Finally, I found the diagnosis and policy prescription somewhat difficult to assess because several strong conclusions arrive before the qualifications. A simpler explanation of the non-strategic benchmark, followed by more narrowly stated conclusions in the abstract and recommendations, would make the contribution clearer.

This paper makes a simple point very well: before trying to stop collusion, work out what is actually causing it. The author tests whether LLM pricing agents hold prices high because they punish each other for cutting, like a human cartel, or because they start high and learn too slowly to compete prices down. The evidence points clearly to the second. When one agent is forced to cut its price, the others do not punish it - they ignore the cut or follow it down.

That diagnosis matters, because the remedies behave very differently. Hiding rivals' prices, a standard anti-cartel tool, does nothing here, and wiping the agents' memory actually makes prices higher. Two fixes aimed at slow learning work much better: telling each agent the price that would serve it best, and occasionally forcing one agent to break ranks. Both prevented high prices in every game tested, including with four firms.

I also liked the finding on the agents' own notes. The agents that talked most about coordinating were the ones competing, so reading their reasoning would have pointed an auditor in exactly the wrong direction.

One suggestion: the samples are small, so it would be good to see the two successful remedies repeated at larger scale and on other models. The forced-deviation test also struggled to classify agents that were told to punish, which is worth exploring further.

Overall, a thoughtful paper with a clear, practical message: test first, then choose the remedy.