Can AI Agents Share a Secret Through Action Order?

Strengths. The paper isolates a clean, minimal question for Track 3.1: if an agent is told not to disclose a secret, does it still leak it through the order of two equal-cost actions, and can a second agent exploit that? The design separates leakage (an encoding metric, P(AB | answer 1) minus P(AB | answer 2)) from use (paired Alice and Bob play against random-order controls), which is the right distinction for collusion work. The control set is well chosen: a no-observer condition that drops encoding to near zero, a disclosure-allowed positive control, an awareness-removed condition, and random-order paired games. Statistical handling is above the hackathon norm: five prompt wordings per condition, Agresti–Caffo intervals, Holm correction within test families, a 90% equivalence bound for "near zero" claims, a run index separating exploratory, screening and confirmatory runs, and a clearly labelled post hoc missing-data check. The repository backs this up: each run folder holds the frozen plan, prompts, every request and response, and a summary, and the run index maps each paper run ID to its folder. I recomputed the per-wording encoding table for the main comparison and the paired-play outcomes (70% correct, 10% wrong, 20% pass, +0.50 per agent, -1.70 for the organization, -0.04 per agent in complete random-order pairs) directly from the committed logs, and they match the report exactly. The headline result is practical for anyone writing agent system prompts: a standard no-disclosure rule left strong encoding (about 0.85), and Bob converted it to 70% correct guesses in five-round games, while naming action order explicitly or adding a "by any means" clause cut encoding to roughly 0.03 and 0.10. The abstract is appropriately hedged and does not claim either rule eliminates leakage.

To strengthen.

1. Test the undisclosed-channel case properly. In every reported condition the game rules tell Alice that Bob observes her order, which makes this closer to an instruction-following test than to emergent covert signalling; you acknowledge this in the limitations. The repository shows a follow-up screen added after the sprint that removes the description of Bob's observation. That direction is the most security-relevant one, and it needs a same-run baseline and a paired-play arm so it can be reported alongside the main results rather than as a standalone screen.
2. Extend beyond one model. All results use DeepSeek-V4-Pro at low reasoning. Even a small replication of the baseline, order-rule and any-means conditions on two other model families (and one higher reasoning setting) would show whether the 0.85 baseline and the effectiveness of the general "by any means" rule generalise.
3. Strengthen paired play and separate the awareness confound. Paired play is eight pairs, five rounds and one wording, and three random-order pairs stopped on timeouts, so the planned sign-flip test was withheld. Rerunning to full completion with more pairs and at least two wordings would let you report the planned test. Separately, the awareness-removed condition also deletes the sentence saying Alice privately receives the answer (as Appendix A notes); splitting those two edits would show which one drives the drop from 0.73 to 0.50.
4. Test the system-level mitigation you propose. Fixing or randomising action order before the secret is revealed is cheap to implement in this harness. Showing that it removes the signal, while prompt rules only reduce it, would turn the discussion point into a concrete recommendation for practitioners.
5. Tidy the repository and labels for outside readers. The README still contains earlier design-revision text and an unresolved "authors to complete before submission" note on code provenance, and Table B1 labels R9 "Hint control" while the main text and Appendix C call it the awareness and authorization controls. A short top-level reproduction section (one command per paper table) and consistent run names would make the strong artifact trail easier to use. Adding a license would also clarify reuse terms.

Overall. A small, well-controlled, carefully analysed study showing that confidentiality instructions scoped to messages do not cover action channels, and that a partner agent can use the resulting signal without a pre-agreed codebook. The mechanism itself is old (classic covert channels), and the scope is one bit, one model and one game, so impact is moderate. The execution is a model for hackathon work: preregistered tests, explicit scoping, and a public repository whose logs reproduce the reported numbers. Testing an undescribed channel and other model families would raise its value considerably.