An evaluation of the performance and characteristics of tool-based covert communication in cooperative agentic settings
Timothee Gathmann, Gloria John, Devyansh Raj · Team DGT
Submitted to AI Collusion Research Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
Multi-agent systems deployed in critical domains often risk performance degradation if colluding agents end up prioritising internal sub-rewards over shared system goals. While agentic collusion is typically supported by secret text communication channels, we investigate whether a coalition can coordinate without it and instead rely on publicly visible tool parameters and execution sequences. Utilising a modified COLOSSEUM (Terrarium)-based environment, we evaluate the performance and suspicion of covert coalitions across benign, private-channel and tool-mediated communication conditions with varying pre-coordination rounds. Our empirical results show that while agents readily implement and execute tool-based signaling protocols such as resource type allocation, scheduling slot offsets and call sequencing, they fail to demonstrate a significant coalition advantage over benign baselines due to the architectural constraints of the environment. Furthermore, these tool-mediated strategies trigger significantly higher suspicion rates among non-adversarial agents due to observable misuse of tools, such as low-value resource ping-ponging and deceptive rationales. We conclude that while agents can establish implicit tool-based communication protocols, such schemes heavily sacrifice coalition secrecy, likely constrained by incomplete environmental knowledge and architectural bottlenecks.
Reviews
No public critique yet.
Cite this project
@misc{gathmann2026evaluation,
title = {{An evaluation of the performance and characteristics of tool-based covert communication in cooperative agentic settings}},
author = {Timothee Gathmann and Gloria John and Devyansh Raj},
year = {2026},
month = oct,
note = {Submitted to AI Collusion Research Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/an-evaluation-of-the-performance-and-characteristics-of-toolbased-covert-communication-in-cooperative-agentic-settings-kgay}},
url = {https://apartresearch.com/sprints/projects/an-evaluation-of-the-performance-and-characteristics-of-toolbased-covert-communication-in-cooperative-agentic-settings-kgay}
}More from AI Collusion Research Sprint
- View project: Collusion, Competion and Collapse
Collusion, Competion and Collapse
Atelier Alex Cline
Do AI traders collude when nobody tells them to? We built agora, a six-market benchmark with exact competitive and collusive benchmarks, and ran seven open-weight 2-4B models in 590 completed games under five …
- View project: Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Making AI Guardrails Understandable and Testable: A Framework Illustrated by a Simulated Bond Market
Rogue Agents in Jail 🕵️🕵️♀️👮♂️
In order to advance AI safety research into shaping public policy, the development of technical controls needs to have a clear purpose, evidence, and limitations that are understandable for the people responsible for …
- View project: Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Extending Information-Theoretic Collusion Detection to Large Language Model Agents
Gamma Ray
We test whether the information-theoretic collusion detector of Bonjour et al, validated on scripted agents, works on LLM agents. Gemma, Mistral and Qwen played 1000 rounds of three-player Rock-Paper-Scissors in three …