Market History Visibility and High Prices in LLM Pricing Agents

This is a carefully scoped case study. It shows that a price-above-Nash screen flags two runs with very different underlying mechanisms. The paper does an unusually good job of separating what was observed from how it is interpreted. It states the history/feedback confound, admits that the analysis window was chosen after inspection, and avoids claiming collusion. The suggested next experiment, hiding rivals’ prices while keeping each agent’s own feedback, is exactly the right one. The concerns are mainly about how much the design can show and about one under-analyzed result.

The no-history result follows directly from the prompt. With memory=0, each round is effectively a fresh one-shot decision. The paper notes that the agent still thinks it is in round 1 at round 30. So this is closer to “one-shot vs repeated game” than to a test of history visibility within a repeated game. The specific price comes straight from the starter framework’s prompt, which tells agents the consumer willingness-to-pay (4.51). Taking the midpoint between cost (1) and that value gives the 2.755 price exactly. The paper should trace the price to that prompt detail. It would then be clear the result is driven by the prompt rather than specific to the model, and that any LLM given the same prompt would likely land near the same price.

The history run deserves more analysis than it gets. In rounds 21–30, its mean profit (30.07) is about 68% of the way from Nash (22.29) to joint-profit (33.75), and its mean price is about 50% of the way. By the profit-based collusion index in Calvano et al. and Fish et al., that is a substantial supracompetitive outcome. The decision records also describe a “tacit convention.” The paper presents this run mainly as a failed convention, based on the final two rounds. With only 30 rounds, compared with the 300 typical in the Fish et al. setup, it can’t tell whether that drop is a breakdown, a temporary price war, or noise. Ironically, the run with lower prices may be the stronger collusion case, and that is the paper’s own thesis. It should say so directly.

There is too little data for the comparison. One run per condition, 30 rounds each, at temperature 0.7, and seed 0 isn’t even passed through to the API, as the authors note. Five to ten runs per condition with longer horizons would make the contrast and the history-run dynamics interpretable. The rounds 21–30 window should be fixed in advance or justified, e.g. by matching standard practice.

Reproducibility. The numbers were “retained from the experiment report and not independently recomputed,” and no raw JSON or Parquet records or launcher script are released. The run IDs document where the results came from, but nobody can check them. Releasing the two run folders would cost little and settle this.

Minor. The keyword list and framing put “collusion auditing” up front, but the paper doesn’t test a detector. The audit checks it proposes (responses to rivals’ prices, sustained joint gains, behavior after deviations) would make a natural follow-up, and the history run is the place to try them.

Suggestions: Run the factorial design the authors already propose (own feedback × rival-price visibility), with several runs per cell and about 100+ rounds. Report both price-based and profit-based indices. Release the raw trajectories.