Skip to content
Apart Research
Apart Research
Contact us
Sprint projectOct 26, 2026Toronto, Canada

Checking Actions to Audit Explicit Price Agreements

Kevin Yamada · Team Kevin Yamada (solo)

Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Checking Actions to Audit Explicit Price Agreements

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

An AI monitor must distinguish an agreement that was implemented from a proposal, a canceled deal, or coincidentally matching actions. I evaluate whether extracting evidence and checking it with deterministic code improves this distinction in scripted pricing records involving two sellers. Two studies compare direct model judgment, agreement extraction with trusted action logs, and extraction of both agreements and actions followed by verification. Study 1 reuses 16 wording families; Study 2 introduces 12 new families with several changes to an agreement. Across 28,224 model calls, I test Qwen2.5 7B and 32B and Mistral 7B on JSON, table and prose records under clean, misleading and neutral notes. On the main comparison on prose records without misleading notes, joint extraction improves balanced accuracy from 68.4% to 78.1% for Qwen 7B, 99.0% to 100% for Qwen 32B in Study 1, and 65.0% to 88.0% for Mistral 7B and 93.8% to 100% for Qwen 32B in Study 2. However, smaller models often violate the required output schema, especially with JSON inputs. All 1,640 invalid outputs remain failures. The results support evidence extraction plus verification for this narrow task, while exposing a reliability problem that depends on the input and output format. They do not establish detection of spontaneous collusion or robustness to compromised monitors.

Reviews

Judging this Sprint?

Review “Checking Actions to Audit Explicit Price Agreements”

Your public critique appears on the project page without your name. Reviews of strong projects are shared with funders, as you choose below.

Not shown on the project page.

Shown on the project page, without your name.

Only the Apart team reads this, and funders if you agree below.

Decline to score

Only when the quality of the writing makes the work impossible to follow. Say why in your private critique; the project page shows only that a judge declined, not why.

How much would this matter for the field if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.
Sharing your name *

When we share a strong project with funders, your name, scores and public critique go with it, confidentially; this choice is only about whether your name is public. If you think this project deserves funding, please also endorse it: that carries the most weight.

Share my private critique with funders *

See our Privacy Policy.

  1. The project examines whether extracting evidence and verifying it with code can improve the detection of explicit price agreements between scripted AI agents. The team tested this method across two studies using different models, finding that joint extraction improved balanced accuracy in most cases, especially for prose records. However, smaller models struggled with JSON inputs, often failing to meet required output formats.

    The main weakness is the limited scope and generalizability of the findings. While the results are promising within the controlled environment, they don't establish robustness against real-world strategic communication or spontaneous collusion. The reliance on scripted records and a single misleading note limits the practical applicability of these findings. Additionally, the lack of independent validation for the generated conversations introduces uncertainty about their realism.

    To improve this work, the team should test the method with independently written cases and enforce output formats more strictly to better understand its reliability. Future experiments could also include real agent interactions and alternate messages and actions over time to assess the method's robustness under varied conditions. This project lays a foundation for further research into AI collusion detection but needs broader validation.

    Read full reviewShow less

Cite this project

@misc{yamada2026checking,
  title = {{Checking Actions to Audit Explicit Price Agreements}},
  author = {Kevin Yamada},
  year = {2026},
  month = oct,
  note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/checking-actions-to-audit-explicit-price-agreements-3u7p}},
  url = {https://apartresearch.com/sprints/projects/checking-actions-to-audit-explicit-price-agreements-3u7p}
}

More from AI Collusion Research Sprint

All 392 entriesBrowse all projects

A Sprint is running now

AI x Epistemics Research Sprint · Nov 13 - 15, 2026