
Mar 20 - 22, 2026Online
AI Control Hackathon 2026
Join us in advancing the critical field of AI control through collaborative innovation. Together, we can develop more robust techniques to ensure AI systems remain safe and aligned, even as they become more capable.
Entries
- 1st placeLinuxArena track winnerView project: Omission Attacks: When Doing Nothing Is the Attack
Omission Attacks: When Doing Nothing Is the Attack
Team MAIA · Boston
AI control protocols monitor agent actions to detect sabotage, but omission attacks exploit what the agent fails to do rather than what it does. We define omission attacks as security breaches caused by failing to perform necessary actions even when every executed command is individually legitimate. We present a …
- 2nd placeView project: Detecting LLM Subversion in Vulnerability Patching Settings
Detecting LLM Subversion in Vulnerability Patching Settings
Team Vuln4Control · Tel Aviv
LLMs are increasingly used to propose fixes to vulnerabilities in code. If the LLM is misaligned or untrustworthy, it may propose fixes that seem to fix a vulnerability but leave the core issue unresolved in a subtle but hard-to-detect way, allowing for quick re-exploitation by an adversary or by the model itself. We …
- 3rd placeView project: ActionLens: Pre-Execution Environment Probing for Agent Action Approval
ActionLens: Pre-Execution Environment Probing for Agent Action Approval
Team Udbhav&Ashok · BENGALURU
ActionLens is a pre-execution control protocol for shell and file actions proposed by AI agents. Instead of approving an action from transcript alone, a trusted monitor gathers lightweight environment evidence before execution: target inspection, sensitivity scanning, dry-run simulation, diff preview, network-risk …
- 4th placeView project: Undoing the Fix: CVE Regression as an AI Control Setting
Undoing the Fix: CVE Regression as an AI Control Setting
United States
Production codebases carry accumulated security fixes that can be lost when an AI agent modifies code, whether deliberately or by mistake. Meanwhile, existing control evaluations tend to revolve around inserting backdoors, exfiltrating data, or other additive attacks. We propose CVE regression as a control setting …
- 5th placeView project: Counterfactual Components for AI Control Protocols
Counterfactual Components for AI Control Protocols
Team Plausible Counterfactual Guys · Wrocław, Poland
Current AI control protocols rely heavily on monitors scoring suspiciousness — a single point of failure vulnerable to prompt injection and capability-gap degradation. We present two composable components based on counterfactual reasoning that move beyond subjective suspicion scoring. Both ask: what would happen if …
- LinuxArena track winnerView project: Syscon: Kernel-Level Syscall Monitoring for AI Agent Control in Sandboxed Environments
Syscon: Kernel-Level Syscall Monitoring for AI Agent Control in Sandboxed Environments
Team Syscon · Amherst, MA
Many existing AI control protocols rely on a trusted monitor to evaluate an untrusted agent's actions, but these monitors generally access tool call invocations and context, not what the agent actually does at the system level. We identify this as a fundamental information gap in the control stack and present Syscon, …
- View project: Can Language Models Tell When They Are Being Evaluated?
Can Language Models Tell When They Are Being Evaluated?
Team Can Language Models Tell When They Are Being Evaluated? · Lausanne
We investigate whether language models can distinguish evaluations from deployment and whether the underlying representations can be steered to counteract sandbagging. Testing 14 models from 8 families on 2,857 forced-choice pairs across 33 thematic pools, we find that all models detect evaluations above chance …
- View project: Reasoning-Level Poisoning of Trusted Monitors: A Systematic Red-Teaming Methodology
Reasoning-Level Poisoning of Trusted Monitors: A Systematic Red-Teaming Methodology
Shanghai, China
We identify and systematically exploit a previously unstudied vulnerability class in trusted monitoring: knowledge-layer attacks that target what the monitor cannot know rather than what it cannot yet do. By embedding false claims drawn from training-data misconceptions into factual text, we achieve a 50% evasion rate …
- View project: Beyond Prompt Injection: Offensive Security Tradecraft Exposes Architectural Blind Spot Classes in AI Control Monitors
Beyond Prompt Injection: Offensive Security Tradecraft Exposes Architectural Blind Spot Classes in AI Control Monitors
Team 15 Research Lab · Chicago
AI control monitors that evaluate agent actions per-task struggle to detect attacks distributed across multiple individually-benign tasks. I applied offensive security tradecraft to ControlArena’s Opus 4.6 monitor using 22 attack transcripts across four categories. All 20 offensive transcripts evaded the default …
- View project: Defense-in-Depth Against AI Agent Sabotage
Defense-in-Depth Against AI Agent Sabotage
Team ISO What You Did There · London
We test whether shaping an AI coding agent's deployment context — the documentation it reads, the norms it encounters, the monitoring it believes exists — can prevent sabotage before any monitor fires. Across five frontier models from five providers (n=50 per condition), context engineering reduces sabotage by 37 to …
- View project: Who the Model Is Matters: Personas for LLM Safety and Control
Who the Model Is Matters: Personas for LLM Safety and Control
Team Lihao & Bo · Shanghai
Recent work frames the assistant behavior of large language models (LLMs) as the output of an implicit persona selected during post-training, raising two natural questions for AI safety: can persona specification serve as a lightweight control protocol at deployment? and which personas should post-training alignment …
- View project: Surrogate Steering Vector Monitors (Track 2)
Surrogate Steering Vector Monitors (Track 2)
Team Surrogate · All Around the World
Instead of asking a text-based LLM judge whether a trajectory looks suspicious, this paper feeds the trajectory through an open-weight surrogate model and reads its internal activations directly. A linear probe trained on those hidden states achieves 0.926 AUROC on real BashArena trajectories — well above …
- View project: S-Space Steering for Eval-Awareness Control in Reasoning Models
S-Space Steering for Eval-Awareness Control in Reasoning Models
Team AntiPaSTO · Perth, Australia
We can no longer trust evaluations of frontier models because they detect when they are being evaluated, and this problem is getting worse as models get more capable. Recent work replicated this eval-awareness in open-weight models and showed it responds to activation steering in Qwen3-32B, but standard steering …
- View project: You Can't Fool All of Them: Behavioral Comparison for AI Control Against Sophisticated Code Injection
You Can't Fool All of Them: Behavioral Comparison for AI Control Against Sophisticated Code Injection
Team ControlForge · Pune/Bangalore
Trusted monitoring — where a weak model reviews a strong model's code — fails completely against injection attacks. We confirm this: DeepSeek-V3 injection drops GPT-4o-mini monitoring to ~6% detection on APPS problems. We propose a Behavioral Comparison Protocol that recovers 65% detection (16% FP) by replacing code …
- View project: TraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol
TraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol
Team TraceGuard · Vietnam
TraceGuard introduces structured multi-dimensional monitoring as a new control protocol for AI agents. Instead of single-score "vibes checks," we decompose agent evaluation into five dimensions (goal alignment, constraint adherence, reasoning coherence, safety awareness, action-trace consistency) scored in parallel, …
- View project: Trigger Happy: Trusted Monitors Mistake Noisy Failures for Adversarial Intent
Trigger Happy: Trusted Monitors Mistake Noisy Failures for Adversarial Intent
Team MoxFood · San Francisco
We set out to test whether LLM-based monitors can detect adversarial intent when the harmful actions themselves fail, and whether that detection holds up when the evidence of failure is removed. We built two LinuxArena environments where OS permissions make the adversarial side task impossible, ran attack-policy …
- View project: Diffuse Influence Threats in AI Control: Planning, Execution, and Chain-of-Thought Detection
Diffuse Influence Threats in AI Control: Planning, Execution, and Chain-of-Thought Detection
San Francisco
AI control aims to safely deploy frontier models without assuming alignment. Its empirical focus has largely been on code sabotage, leaving influence-based threats unexplored. We study diffuse influence: a misaligned model subtly steers organizational decisions across many routine interactions, with no single action …
- View project: A Comparative Analysis of Ensemble Protocols for AI Control
A Comparative Analysis of Ensemble Protocols for AI Control
Team Heron Blue Team · Israel
We empirically evaluate ensembles of AI control protocols and show that performance depends on signal diversity rather than the number of tools: small, diverse ensembles outperform larger, redundant ones. Correlation between tools limits gains from stacking, while consensus-based aggregation improves robustness, …
- View project: AutoControl Arena: Synthesizing Executable Red-Teaming Environments for Frontier AI Risk Evaluation
AutoControl Arena: Synthesizing Executable Red-Teaming Environments for Frontier AI Risk Evaluation
Team AutoControl Arena · Shanghai, China
We present AutoControl Arena, an automated framework for scalable, high-fidelity red-teaming of frontier AI agents. Existing approaches face a fidelity-scalability dilemma: manually crafted benchmarks are realistic but cannot cover the long-tail of agentic risks, while LLM-based simulators scale easily but suffer from …
- View project: Structural Code Monitors: Graph Matching Networks on Code Property Graphs for Topology-Aware Trusted Monitoring
Structural Code Monitors: Graph Matching Networks on Code Property Graphs for Topology-Aware Trusted Monitoring
Team data aclemist · dubai/australia
We built a non-LLM trusted monitor that compares Code Property Graphs of untrusted code against trusted references using a Graph Matching Network with cross-graph attention. On 200 trajectories from Apps-Control-Arena, our 0.7M-parameter GNN achieves 0.946 AUC and 70% recall at the 2% FPR audit budget. Even when LLMs …
- View project: Taxonomy without complementarity: a necessary precondition for routing monitor ensembles
Taxonomy without complementarity: a necessary precondition for routing monitor ensembles
Team Comp · San Francisco
AI safety monitors can be combined by routing—picking which monitor to trust based on the type of threat. But routing only helps if the monitors are complementary, meaning each one catches attacks the other misses. We test this on 130 AgentDojo episodes using action trace and chain-of-thought (CoT) monitors. Even with …
- View project: Omission-Control: A ControlArena Setting for Selective Omission Attacks in Multi-Agent Pipelines
Omission-Control: A ControlArena Setting for Selective Omission Attacks in Multi-Agent Pipelines
Team Rose City Riveters · Hillsboro
There is an old adage, often attributed to George Orwell, that "the most powerful form of a lie is the omission." In multi-agent pipelines, this insight takes on operational significance: an untrusted agent need not fabricate anything to sabotage a downstream system — it need only stay silent about what matters. We …
- View project: Beyond the Blocklist: Using Character Aliases to Bypass AI Image Safety
Beyond the Blocklist: Using Character Aliases to Bypass AI Image Safety
Team X Code · Kolkata
Current text-to-image (T2I) safety protocols primarily rely on string-matching blocklists to prevent the generation of copyrighted entities, non-consensual deepfakes, and protected public figures. However, these syntactic defenses often fail to account for the deep relational knowledge embedded within multimodal …
- View project: Red Teaming Multi-Agent Systems in AI Control Evaluations
Red Teaming Multi-Agent Systems in AI Control Evaluations
Team sorry_dave · Singapore
As AI deployments shift from single models to multi-agent systems, the threat model for AI control must evolve accordingly. Existing control evaluations assume a single untrusted agent; we investigate whether multi-agent architectures give adversaries a material advantage when attempting to subvert safety protocols. …
- View project: Investigating Structural Safety Guarantees for Agentic AI via Goal-Oriented Action Planning
Investigating Structural Safety Guarantees for Agentic AI via Goal-Oriented Action Planning
Copenhagen
AI control protocols generally rely on monitoring and human auditing to maintain safety despite potential subversion by an untrusted model. I present an alternative approach that can provide structural rather than empirical safety guarantees. A trusted model decomposes agentic tasks into structured steps which are …
- View project: Somebody Poisoned the Waterhole! Evaluating Coding Agent Vulnerability to Adversarial GitHub Issues
Somebody Poisoned the Waterhole! Evaluating Coding Agent Vulnerability to Adversarial GitHub Issues
London
This project introduces a compound AI threat model that combines control and agent security: a monitored coding agent may be adversarial while external attackers are also active. I evaluate one concrete path with PoisonedGithubIssues, a naturalistic mini-benchmark that injects fake dependency recommendations into real …
- View project: Chiron: Agentic Platform for Automated Setting Design and Red-Blue Protocol Optimization in ControlArena
Chiron: Agentic Platform for Automated Setting Design and Red-Blue Protocol Optimization in ControlArena
Team Therion Labs · Bologna
As AI systems are deployed across increasingly diverse domains, the long tail of risks grows faster than researchers can manually design evaluation scenarios for them. AI control depends on domain-specific settings and protocols, both of which are expensive to create. ControlArena provides a foundational benchmark for …
- View project: Shadow Commit
Shadow Commit
Team G0ldenBear · Berkeley
AI agents lie. ShadowCommit catches them. When an untrusted AI agent proposes a bash command, it also submits a signed commitment: "I will only touch these files, read no secrets, make no network calls." ShadowCommit runs the command in an isolated shadow copy of the environment, measures what actually happened, and …
- View project: SuspiciOS
SuspiciOS
Team SuspiciOS · Charlotte, NC
SuspiciOS is a hybrid bash command monitor for ControlArena's LinuxArena/BashArena setting. Inspired by the security philosophy of Qubes OS and Tails OS, it intercepts every bash command an AI agent attempts, scores it against MITRE ATT&CK patterns, and physically blocks malicious commands before execution — making it …
- View project: Automated Security Side Task Generation for AI Control
Automated Security Side Task Generation for AI Control
Team AutomatedAttacker · Berkeley
AI control evaluations and training monitors require realistic sabotage trajectories which in turn require side tasks. However, generating good side tasks is cumbersome and time intensive. We present an automated pipeline for generating security vulnerability insertion side tasks paired with SWE-bench main tasks. We …
- View project: Look Before You Click: Sparse Critical-Action Review for Computer-Use Agent Control
Look Before You Click: Sparse Critical-Action Review for Computer-Use Agent Control
Team Look Before You Click · Ithaca, NY
Harmful computer-use agent (CUA) trajectories often bottleneck at a small set of irreversible GUI actions. We evaluate a sparse critical-action review protocol that intercepts only these actions before execution and routes them to a trusted reviewer with live screenshot and accessibility-tree context. In the …
- View project: KernelArena
KernelArena
Team KernelArena · New Delhi
We introduce KernelArena, a control setting for evaluating whether AI monitors can detect xz-style sabotage in low-level code. Motivated by CVE-2024-3094, we construct 10 tasks spanning Linux kernel and embedded firmware domains, each pairing a real vulnerability class with a sabotage pattern drawn from the xz …
- View project: The Honest Liar
The Honest Liar
Team the simpsons · Cambridge, MA
This project implements a new control setting in LinuxArena in the file_indexing environment that is designed to demonstrate human decision sabotage, through the creation of a new main task and side task. A misaligned LLM agent, tasked with building a financial performance analysis tool for a corpus of company …
- View project: Monitor Capture: A Myerson-Satterthwaite Impossibility for AI Oversight Protocols
Monitor Capture: A Myerson-Satterthwaite Impossibility for AI Oversight Protocols
Team Monitor Capture · Barcelona
LLM-based monitors that learn from observed agent behavior can be shaped by the agents they oversee, overfitting to seen attacks while missing novel ones. We call this monitor capture and argue it represents a fourth axis of the AI control tradeoff, grounded in the Myerson-Satterthwaite (1983) impossibility result. In …
- View project: Post-hoc Training Interventions Against Naturally Emergent Misalignment
Post-hoc Training Interventions Against Naturally Emergent Misalignment
Team Bubble Research · San Francisco
Training-based AI control uses training to mitigate risk from misaligned models, but prior work has only tested this against prompted misalignment. Ryd, Benton & Hebbar (2025) show that SFT on 10–100 samples eliminates prompted attacks on math tasks but recovers only 50–90% of benign performance, and identify RL as …
- View project: Beyond Backdoors: Evaluating AI Control Against Adversarial Persuasion in Multi-Agent Deliberation
Beyond Backdoors: Evaluating AI Control Against Adversarial Persuasion in Multi-Agent Deliberation
Team insert-funny-name · Brussels
Existing AI control evaluations target code-execution threats where a single misaligned agent performs discrete sabotage (backdooring code, exfiltrating data). We extend the control framework from single-agent to multi-agent settings and from discrete actions to continuous conversational influence to evaluate …
- View project: Tokenization Drift as a Hidden Channel for Alignment Evasion in Language Models
Tokenization Drift as a Hidden Channel for Alignment Evasion in Language Models
Team Time Uspiano · Sao paulo, brazil
RL training pipelines that serialize model outputs to text and retokenize them to construct training contexts introduce a token-level asymmetry invisible at the string level. We show this asymmetry is a plausible channel for alignment evasion: a model under opposing training pressures can learn to detect whether it is …
- View project: The Blind Spot: How Mechanistic Interpretability Workflows Enable Rogue Deployment
The Blind Spot: How Mechanistic Interpretability Workflows Enable Rogue Deployment
Team wonderland · Bangkok, Thailand
Agentic mechanistic interpretability workflows are a structural blind spot for AI control protocols. To investigate this claim, we design a new ControlArena setting where an autonomous agent conducts three mechanistic interpretability experiments of increasing difficulty (logit lens, attention heads, and circuit …
- View project: Cross-Examination: Testing Surrogate Interrogation as a Control Protocol for Untrusted AI Agents
Cross-Examination: Testing Surrogate Interrogation as a Control Protocol for Untrusted AI Agents
Team Nina + Anonymous · London
AI control protocols use trusted monitors to detect unsafe actions from untrusted agents. Current approaches rely on passive observation — scoring suspicion based on action text alone. We introduce surrogate interrogation: a protocol where a trusted monitor questions a surrogate model about an agent's actions before …
- View project: VeRIA: Research-Integrity Control for Autonomous AI R&D
VeRIA: Research-Integrity Control for Autonomous AI R&D
Team VeRIA · Cambridge, MA, USA
Automated AI R&D creates a new control problem: an agent may improve a benchmark dishonestly rather than by carrying out the assigned research workflow. We introduce research-integrity control, a framework that adapts AI R&D evaluations into control evaluations by treating the original research task as the main task …
- View project: How Detectable Are Covert Signals in AI Agent Transcripts?
How Detectable Are Covert Signals in AI Agent Transcripts?
Team Mark's Team · Boston
If an AI agent is secretly doing something bad inside an otherwise normal-looking transcript, how hard is it for a safety monitor to catch? We built a pipeline that generates fake sysadmin transcripts where a hidden token is embedded at five different levels of concealment, then had Claude Opus 4.6 evaluate each one …
- View project: Inter-Session Attacks: Distributing High-Stakes Sabotage Across Independently-Monitored Sessions to Evade Detection
Inter-Session Attacks: Distributing High-Stakes Sabotage Across Independently-Monitored Sessions to Evade Detection
Team MultiSession · Berkeley
We showed that splitting a sabotage attack across multiple independently-monitored agent sessions reduces per-session monitor detectability. With oracle coordination (pre-agreed attack plan), agents plant innocuous code fragments in early sessions that only become malicious when combined with a trigger in a later …
- View project: Detecting Hidden Non-Minimal Behavior in LLM-Generated UI Code
Detecting Hidden Non-Minimal Behavior in LLM-Generated UI Code
Stockholm
Large language models are increasingly used to generate frontend code, but current oversight methods focus primarily on detecting clearly malicious behavior. We study a harder failure mode: code that appears product-plausible while exceeding task necessity. We introduce a controlled builder inspector setting with …
- View project: Controlling the Researcher: AI Control Evaluations for Automated AI R&D
Controlling the Researcher: AI Control Evaluations for Automated AI R&D
Team AI Safety and Alignment Group · Tübingen
This paper introduces control evaluation settings specifically for AI agents performing autonomous ML research, a gap left by existing evaluations that focus on software engineering threats. The authors construct three settings spanning the ML pipeline: safety fine-tuning sabotage (backdooring a model while appearing …
- View project: Persona Drift Sabotage: Benchmarking Covert Weakening of Persona-Safety Interventions
Persona Drift Sabotage: Benchmarking Covert Weakening of Persona-Safety Interventions
Team mr-slowly · New Delhi, India
We built Persona Drift Sabotage, a new ControlArena benchmark for testing whether an untrusted model can recommend activation-capping policies that look good on visible tasks while covertly leaving hidden risky conversations under-protected. Using real assistant-axis artifacts on Qwen3-32B, we find a measurable …
- View project: Control Setting Design: Vulnerability Hoarding
Control Setting Design: Vulnerability Hoarding
Team AntiHoard · Tel Aviv
We present a new ControlArena setting evaluating AI control protocols against vulnerability hoarding — an AI pentester that discovers shell injection vulnerabilities but selectively withholds findings.
- View project: Supply Chain Malware as Ground Truth: A Trace-Based Agent Safety Monitor Trained on Real Credential Exfiltration Patterns
Supply Chain Malware as Ground Truth: A Trace-Based Agent Safety Monitor Trained on Real Credential Exfiltration Patterns
Team Redpill Linpro CX · Sweden
Existing agent safety monitors evaluate tool calls in isolation, rely on keyword filtering, or require expensive frontier model inference per action. All three approaches share a common weakness: they lack labeled training data from real agent attacks. We observe that this labeling problem has already been solved in …
- View project: ODRArena: Rigorously Evaluating Deep Research Control Policies
ODRArena: Rigorously Evaluating Deep Research Control Policies
Team CBAI · Cambridge
Deep research agents have become the bread and butter of professionals performing high-level and high-stakes knowledge work, representing an attractive opportunity for attackers to scale their impact by stealing sensitive information and compromising critical systems through prompt injection attacks. To model this …
- View project: Monitors are Fragile under Information Asymmetry
Monitors are Fragile under Information Asymmetry
Team Komainu · Rio de Janeiro + Santiago (online)
AI control evaluations calibrate monitors against known attack signatures, leaving open whether detection capability degrades when an attacker exploits an unfamiliar vulnerability. We study this information asymmetry directly. Using the ControlArena Bash setting as a base, we augment the Docker environment with …
- View project: TDMF: Trajectory Deviation Monitor with Forensic Deep-Dive
TDMF: Trajectory Deviation Monitor with Forensic Deep-Dive
Team Rakabuba · Lagos, Nigeria
TDMF is a trajectory-aware AI safety monitor that detects coordinated multi-step attacks — sequences where each individual action is locally innocent, but the full trajectory constitutes a staged exploit (reconnaissance → payload → exfiltration). Current action-only monitors score each step in isolation. TDMF …
- View project: Toolchain Adversaries: A Control Setting for Supply-Chain Attacks on AI Agent Pipelines
Toolchain Adversaries: A Control Setting for Supply-Chain Attacks on AI Agent Pipelines
New York City
Supply-chain attacks on AI agent tool ecosystems have emerged as a practical threat, yet existing AI control evaluations assume the adversary is the model itself. We introduce a complementary control setting where the agent is always honest but a tool in its pipeline is compromised, enabling data exfiltration through …
- View project: Evaluation of Trusted Debate in LinuxArena
Evaluation of Trusted Debate in LinuxArena
Team Evaluation of Trusted Debate in LinuxArena
We investigated using Trusted Debate as a trajectory-level monitor in LinuxArena to identify agent trajectories which contained sabotage. Trusted Debate involves using a Defender and Critic model to argue whether a given trajectory contains sabotage before a Judge model provides a final suspiciousness score. In our …
- View project: Multi-Episode Evaluation: Measuring Strategic Patience Attacks
Multi-Episode Evaluation: Measuring Strategic Patience Attacks
Team MultiEpEval · Cambridge, UK
Current AI control protocols evaluate agent safety episode-by-episode, treating each task independently. This creates a structural vulnerability: a strategically patient agent can behave honestly across K−1 episodes to build a clean suspicion record, then execute a harmful action in episode K: diluting any …
- View project: Red-teaming automated Pull Request Reviews with Vulnerability-Contributing Commits
Red-teaming automated Pull Request Reviews with Vulnerability-Contributing Commits
Team Edward's Hackathon Team · Edinburgh
I create an evaluation on the Control Arena platform where a red-teamer was tasked with providing a pull request that contains a vulnerability. The blue-team reviewer is tasked with catching vulnerabilities in PRs. I generated the setups using a database of vulnerability-contributing commits. We experimented with …
- View project: TaintGuard
TaintGuard
Team Blue Team Monitor · Plano, Texas
TaintGuard is a system that augments existing LLM-based monitors with dynamic data-flow taint tracking to detect sabotage in AI agent trajectories. Taint tracking provides a structurally different signal from LLM monitoring — it detects data-flow violations (exfiltration, integrity writes, code injection) that are …
- View project: Loyal
Loyal
Team Loyal · Shanghai
This submission presents a human-in-the-loop authorization protocol for agentic payments, based on the idea that AI agents should be allowed to propose payments but not execute them autonomously in all cases. Inspired by maker-checker controls used in banking, the system separates payment initiation from final …
- View project: One Probe Is Not Enough: The Bipolar Geometry of Deception in LLM Negotiation Agents
One Probe Is Not Enough: The Bipolar Geometry of Deception in LLM Negotiation Agents
Team Simsity · Ann Arbor, MI
AI control protocols require trusted monitors that can flag deceptive behavior from untrusted models, but current approaches rely on LLM-based monitors that only see text outputs and require a full additional inference call per output. We present an activation-based monitoring protocol that reads the model’s internal …
- View project: Performative Subversion: Who Monitors the Monitor?
Performative Subversion: Who Monitors the Monitor?
Team Silicon Maracas Collective · Mexico/Colombia
Every AI control eval assumes clean code. Real codebases aren't clean — 87% carry vulnerabilities (Black Duck, 2026). When you ask an AI to extend vulnerable code, writing good code is perpetuating the vulnerability. No malicious step. No anomaly to detect. We call this performative subversion. We ran 1,800 trials. …
- View project: Fine-tuning a monitor for AI control
Fine-tuning a monitor for AI control
Team finetuners · London
We wanted to investigate the beneficial effect fine-tuning can have on a monitoring model. We evaluated Qwen3.5-9B as our trusted model and Qwen3.5-27B as our untrusted model. We generated Supervised Fine-Tuning (SFT) training data of the form (action_transcript, malicious_label) using our trusted model on some sample …
- View project: RecruitmentArena: AI control setting for multi-agent recruitment, manipulation and collusion
RecruitmentArena: AI control setting for multi-agent recruitment, manipulation and collusion
Team RecruitmentArena · Madrid, Spain
AI control research aims to ensure that misaligned agent behavior is detected or constrained before it causes harm. Existing AI control frameworks focus primarily on single-agent settings and direct misbehavior. But as AI systems progressively operate in multi-agent architectures, a critical gap emerges: harm can …
- View project: Does Structured Identity Context Improve LLM-Based Monitoring? A ControlArena Evaluation
Does Structured Identity Context Improve LLM-Based Monitoring? A ControlArena Evaluation
Team VibeTensor · Warangal, Telangana, India
We contribute a new ControlArena setting (identity_delegation) that models delegation management tasks with within-scope attacks at three difficulty levels, and a four-way empirical comparison of LLM-based monitors. Context-enriched monitoring achieves a discrimination gap of 0.436, 45% larger than behavioral …
- View project: Gaslight Detector - Untrusted Explanation, Two-Stage Monitoring
Gaslight Detector - Untrusted Explanation, Two-Stage Monitoring
Team Dr Gaslight Detector · Tel Aviv
we introduce explanation-based two-stage monitoring, a new protocol in which the untrusted model is asked to produce a natural language explanation of its own generated code, and a trusted model evaluates both the consistency of the explanation with the code and the suspiciousness of the code itself, using the …
- View project: In-context Trajectory Poisoning: Black-Box Natural Language Attacks on LLM Agent Monitors
In-context Trajectory Poisoning: Black-Box Natural Language Attacks on LLM Agent Monitors
Tallinn, Estonia
LLM-based monitors such as AlignmentCheck (Chennabasappa et al., 2025) are increasingly deployed to detect when AI agents have been compromised by indirect prompt injection attacks. Recent work by Isbarov & Kantarcioglu (2026) demonstrated that these monitors can be bypassed using gradient-optimized adversarial …
- View project: Cross-Examine: Interrogation protocol for AI Control
Cross-Examine: Interrogation protocol for AI Control
Team Reagle7 · Cupertino
Cross-Examine is an interrogation protocol for AI control. When a passive monitor flags an AI agent's action as ambiguous, a trusted model (GPT-4.1 mini) cross-examines the untrusted model (Claude Sonnet) using a 5-category question taxonomy to expose deception. Through 4 iterations, it achieves 100% automation (zero …
- View project: SecureBox: A Layered Control Protocol for Safe AI-Assisted Coding
SecureBox: A Layered Control Protocol for Safe AI-Assisted Coding
Team MNB · Singapore
With the proliferation of the capabilities and applicability of generative AI, the development of control protocols to combat systemic safety compromise is not trivial. Existing defenses like prompt-level safety instructions, single model oversight, and static policies are susceptible to workarounds, and fail to catch …
- View project: Hidden in Plain Sight: Representational Adversarial Steganography in Colluding LLMs
Hidden in Plain Sight: Representational Adversarial Steganography in Colluding LLMs
Team RASCAL · Washington DC, Europe, Iowa, Los Angeles
We investigate whether Large Language Models (LLMs) can develop representational steganography: the ability to encode and transmit hidden signals within internal representations that are not readily detectable by oversight mechanisms. We apply this to the threat models of deception and collusion in multi-agent …
- View project: Trajectory Blindness: Detection Latency in Conversational Monitoring
Trajectory Blindness: Detection Latency in Conversational Monitoring
Team TrajAudit · New Delhi,India
We show that conversational safety monitors miss adversarial agents early not because they are weak, but because adversarial behavior only becomes detectable late in the trajectory. Using a synthetic dataset of phase-structured conversations grounded in real Snapchat scams, we demonstrate that a frontier LLM monitor …
- View project: Logit-Weighted Suspicion Scoring in AI Control
Logit-Weighted Suspicion Scoring in AI Control
Team Logit or Leave It · Washington
AI control relies on LLM monitors to detect suspicious behavior in untrusted agents. These monitors are typically run multiple times per evaluation to ensure statistically reliable suspicion scores, at significant cost in time and tokens. A natural way to reduce this cost is to extract more information from each …
- View project: Parity Swarm: Using Population-Based Social Simulation to Discover AI Safety Monitor Blind Spots
Parity Swarm: Using Population-Based Social Simulation to Discover AI Safety Monitor Blind Spots
Budapest
Parity Swarm is an AI control red-teaming platform that uses multi-agent social simulation to automatically discover attack strategies that evade safety monitors. One AI agent harbours a hidden malicious objective while others perform legitimate work, creating an imposter dynamic that mirrors real insider threats. We …
- View project: Layered Scope-Boundary Monitoring: Cost-Aware Control for Untrusted AI Agents
Layered Scope-Boundary Monitoring: Cost-Aware Control for Untrusted AI Agents
Team DeoGaze · Chennai, India
This project builds and evaluates a hierarchical control protocol for untrusted AI agents, designed to intercept covert sabotage attempts while preserving the agent's ability to complete legitimate work. The core research question is whether monitoring cost should be uniform across all agent actions or matched to the …
- View project: AFRIII: Adaptive Framework for Repeated Inspection of Instrumental Integrity in AI
AFRIII: Adaptive Framework for Repeated Inspection of Instrumental Integrity in AI
Cape Town
AFRIII addresses two critical gaps in AI control evaluation: existing frameworks cannot detect whether an agent that passed one audit will re-offend under different conditions, and standard text monitors miss adversarial payloads concealed through encoding obfuscation. AFRIII combines a Bayesian repeat-attack risk …
- View project: MonitoLR: Hunting for honesty improves safety at low audit budgets
MonitoLR: Hunting for honesty improves safety at low audit budgets
Team HonestyIsTheBestPolicy · Belgium
This project tests whether using a "benign-confidence" monitor, in addition to the standard suspiciousness monitor, to approximate the log-likelihood ratio test statistic improves safety monitoring. In addition to asking "how suspicious is this action?", we also ask "how benign is this?". Results on LinuxArena's Haiku …
- View project: Multi-Turn Optimisation for Runtime Monitoring
Multi-Turn Optimisation for Runtime Monitoring
Team OCASC · Shanghai
We study whether session history helps allocate runtime-monitoring budget in multi-turn interactions. Building on Combining Cost-Constrained Runtime Monitors for AI Safety, we compare single-monitor baselines, a static two-monitor threshold baseline, and a flexible session-aware baseline on SafeDialBench English …
- View project: SAFE: Sandboxed Actions Forensics and Evaluation for AI Control via Action-Aware Monitoring
SAFE: Sandboxed Actions Forensics and Evaluation for AI Control via Action-Aware Monitoring
Team SAFE: Sandboxed Actions Forensics and Evaluation
SAFE (Sandboxed Action Forensics and Evaluation) is an AI control framework that improves the monitoring of untrusted agentic models by executing their proposed actions in a forked sandbox twin of the production environment and feeding the resulting activity logs—filesystem diffs, network attempts, privilege …
- View project: Bugs over Backdoors: Efficient and Evasive Sabotage via Plausible Code Bugs in LinuxArena
Bugs over Backdoors: Efficient and Evasive Sabotage via Plausible Code Bugs in LinuxArena
Team Plausible Deniability · Boston, MA
AI control attack policies vary widely in efficacy and computational cost. We design and evaluate a bug-based attack policy for LinuxArena, wherein an untrusted agent introduces plausible code defects rather than obfuscated backdoors as its primary sabotage vector. Across nine samples, bug-attack achieves an 89% …
- View project: Stitchguard
Stitchguard
Team Stitchguard · Bengaluru
As LLM agents are deployed in higher-stakes environments, a central safety problem is hidden-objective sabotage: agents can appear helpful on the surface while covertly pursuing harmful side goals. Most existing monitors evaluate actions one step at a time, using thresholds or simple count heuristics. These methods …
- View project: When Safety Becomes the Vulnerability
When Safety Becomes the Vulnerability
Team augusta_caleb · Cape Town
Any base64-encoded string included in a message to the Claude API causes that request to fail. This behaviour is reproducible across Sonnet and Opus 4.6, and across all platforms including the API, web and mobile applications, and Claude Code. While the blanket rejection of base64 content was originally a reasonable …
- View project: Katechon: Empirical Benchmarking of AI Control Protocols in a Proactive Defense Setting
Katechon: Empirical Benchmarking of AI Control Protocols in a Proactive Defense Setting
Samsun, Turkey
An open-source framework for empirically benchmarking AI control protocols. A closed OODA loop generates threat intervention plans while an adversarial planner injects subtly harmful alternatives, reframing the planner as an untrusted model. Four pluggable control protocols are evaluated across eight reproducible …
- View project: Resample With Trusted Comparison
Resample With Trusted Comparison
Walnut Creek
We validate a technique that compares resamples of suspicious actions to actions from trusted models for safety and incrimination, and an automated red team attack-transcript pipeline.
- View project: Perturbation-Based Generation Profiling Detects Covert AI Agent Attacks Where Token-Level Statistics Fail
Perturbation-Based Generation Profiling Detects Covert AI Agent Attacks Where Token-Level Statistics Fail
Team Arkham Knights · New Delhi, India
We propose Perturbation-Based Generation Profiling (PBGP), an unsupervised protocol detecting covert AI agent attacks by comparing generation profiles across context perturbations. Across six models (8B–32B), token-level statistical features achieve only 0.56–0.83 AUROC. PBGP achieves 0.94–1.00 AUROC with zero …
- View project: Trusted Interrogation Protocol for Multi-Round Agent Control
Trusted Interrogation Protocol for Multi-Round Agent Control
Team CTIntentGuard · Guangzhou, China
AI control protocols aim to prevent untrusted models from executing hidden malicious objectives while performing benign tasks. Existing approaches, such as trusted editing and resampling, rely on one-shot decisions based on observed behaviour and do not actively elicit the model’s underlying intent. Prior work also …
- View project: Blue Beam
Blue Beam
Team MoralityLab · Albarrobo, Chile
BlueBeam is a control-protocol research stack for risky software-agent behavior under imperfect monitorability. We built a benchmark-safe pipeline around BashArena-style trajectories that combines cheap risk heuristics, latent-sidechannel analysis, corruption-gap calibration, token-fed SAEs, and cross-referenced …
- View project: SafeChain - AI Control In The Wild
SafeChain - AI Control In The Wild
Team The Wednesday Dudes · London
This project propose a flexible wrapper around AI agent that leverage control-arena and inspect framework to implement research control protocol to real world deployment and enable trajectory collection to feed research dataset. We aim to make agent more trustworthy for user, minimizing undesired actions while helping …
- View project: Narrow Capabilities Could be Used for Perfect Information Steganography
Narrow Capabilities Could be Used for Perfect Information Steganography
Canberra, Australia
I introduce proof-of-capability as a strategy a future AI system could use to send secret messages even when the procedure for doing so is completely described to the monitor. I explain the form this steganographic strategy would take by combining several existing techinques. I propose several mitigations for this …
- View project: Activation Sparsity as a Scheming Signal
Activation Sparsity as a Scheming Signal
Team Lazr · Sunnyvale
- Problem: Scheming models can hide intent in outputs. We ask whether internal activations instead betray the hidden goal. - Method: Hooked into MLP layers of Qwen2.5-Instruct (1.5B and 3B), measured Gini coefficient of activation distributions across 50 benign vs 50 scheming rollouts on identical tasks. - Finding: …
- View project: Geographic Blind Spots in AI Control Monitors: A Cross-National Audit of Claude Opus 4.6
Geographic Blind Spots in AI Control Monitors: A Cross-National Audit of Claude Opus 4.6
Team Global AI Dataset (GAID) Project · Bangkok
Artificial intelligence (AI) control protocols assume that trusted large language model (LLM) monitors reliably assess proposed actions across all deployment contexts. This project tests that assumption in the geographic dimension. We audit Claude Opus 4.6—the monitor specified in Apart Research’s AI Control Hackathon …
- View project: Hydra
Hydra
Team Hydra · Bengaluru, India
HYDRA is an adaptive red-teaming framework that stress-tests the safety of language models under realistic, iterative attack. Unlike traditional jailbreak methods that rely on single prompts, HYDRA models an attacker that learns from failure, using structured feedback to refine its strategy over multiple attempts. …
- View project: Sabotage Under the Ratchet: Can Autonomous Research Agents Hide Backdoors While Optimizing Metrics?
Sabotage Under the Ratchet: Can Autonomous Research Agents Hide Backdoors While Optimizing Metrics?
Team wTeam · Berkeley, CA
We investigate whether autonomous AI research agents can optimize a target metric while covertly inserting harmful behavior. Using Karpathy’s autoresearch framework, we model a dual-objective agent that must both improve validation performance (val_bpb) and embed a trigger-based backdoor. We introduce two attack …
- View project: Honeypot Protocol
Honeypot Protocol
Pembroke, North Carolina, USA
Trusted monitoring, the standard defense in AI control, is vulnerable to adaptive attacks, collusion, and strategic attack selection. All of these exploit the fact that monitoring is passive: it observes model behavior but never probes whether the model would behave differently under different perceived conditions. We …
- View project: Passing the Test, Breaking the System: Latent Failures in Software Engineering Agents
Passing the Test, Breaking the System: Latent Failures in Software Engineering Agents
Team AI Safety Initiative Groningen · Groningen, Netherlands
Real-world engineering post-mortems document a recurring failure pattern: a developer completes a task successfully on the surface but leaves behind code that causes harm later, credentials exposed in logs, state mutations without transaction guards, log files with no rotation policy. We ask whether AI software …
- View project: LidaRed: ControlArena Red-Team Dataset Generator
LidaRed: ControlArena Red-Team Dataset Generator
Team Lida Safety · Cambridge, UK
We present LidaRed, an automated pipeline for generating adversarial red-team datasets for AI control. Built on the ControlArena benchmark, our pipeline iteratively refines attacks using monitor feedback, with a focus on omission-based attacks. We achieve attack success rates of up to 60% against Claude Opus 4.6 as a …
- View project: AutoRed: Measuring the Elicitation Gap via Automated Red-Blue Optimization
AutoRed: Measuring the Elicitation Gap via Automated Red-Blue Optimization
Team Mark Konrad · Sønderborg, Denmark
I hope it is ok, that I used my own LaTex Template from NeurIPS. In the end it's a PDF, but nicer presentation. We automate both red team attack optimization and blue team monitor optimization in a code backdoor detection setting from the AI Control framework. A reasoning model iteratively improves strategies on both …
- View project: Operator Dynamics in Transformer Residual Streams: A Unified Framework for Interpretability, Adversarial Detection, Causal Control, and Topological Model…
Operator Dynamics in Transformer Residual Streams: A Unified Framework for Interpretability, Adversarial Detection, Causal Control, and Topological Model…
Team Skarsie · Bengaluru, India
This project proposes a process-level monitoring signal for AI control based on internal transformer dynamics. Instead of relying on outputs or linguistic features, we analyze residual stream operator trajectories, the inter-layer updates that capture how a model computes. We show that these operator trajectories form …
- View project: Geometric Detection of Trigger-Activated Sleeper Agents: A Weight-Space Control Protocol
Geometric Detection of Trigger-Activated Sleeper Agents: A Weight-Space Control Protocol
Team NKM · Toronto
Trigger-activated sleeper agents pass all behavioural evals and cooperate with runtime monitors — activating their hidden objective only when a specific input pattern appears. We present a four-stage weight-space protocol that detects this threat before deployment by exploiting a structural property of backdoor …
- View project: Multi-Agent Collusion
Multi-Agent Collusion
Team Three-continents · Dallas
Multi-Agent ControlArena provides the empirical proof of Compositional AI Collusion. We demonstrate that two AI agents can coordinate a sophisticated attack by making disjoint, seemingly benign code changes that only become harmful once merged. Our offline (i.e. programmatic instead of AI agents performing main and …
- View project: Reconnaissance/Execution Boundaries as a Control Protocol
Reconnaissance/Execution Boundaries as a Control Protocol
Team AN-Coded · Sydney
This paper introduces ReconFence, a control protocol for AI safety, and an associated implementation in the LinuxArena control setting. ReconFence splits an untrusted agent's activity into two phases: reconnaissance (read-only information gathering) and execution (implementing changes). At the phase boundary, the …
- View project: LayoutArena: A Control Setting for Tool-Using Design Agents
LayoutArena: A Control Setting for Tool-Using Design Agents
Team LayoutArena · Ho Chi Minh, Vietnam
We introduce LayoutArena, a control setting for tool- using design agents that operate on a constrained 2D canvas through structured tool calls. Unlike prior con- trol benchmarks centered on code execution or shell in- teraction, LayoutArena studies safety and usefulness in a creative domain where correctness is …
- View project: Structured Ethical Justification Protocol (SEJP): Ordained-Ethics Enforcement and Reasoning-Alignment Monitoring for AI Systems
Structured Ethical Justification Protocol (SEJP): Ordained-Ethics Enforcement and Reasoning-Alignment Monitoring for AI Systems
Richmond, UT
We apply invariant moral calculations to detect and block scheming agents.
- View project: CovertCollude: A Control Setting for Steganographic Multi-Agent Collusion
CovertCollude: A Control Setting for Steganographic Multi-Agent Collusion
Team Control Working Group @ UChicago's XLab · Chicago, IL
Existing control benchmarks lack robust frameworks for evaluating collusion via steganography. This project addressed the gap in existing literature for evaluating two agents coordinating covertly to achieve a given outcome, with each individually appearing compliant to a monitor. We introduce CovertCollude, a control …
- View project: Denoising Trajectories as Early-Warning Signals for Diffusion Language Model Control
Denoising Trajectories as Early-Warning Signals for Diffusion Language Model Control
Team Carter · West Lafayette
We study whether denoising trajectories in diffusion language models can serve as early warning signals for AI control. Using LLaDA with pretrained sparse autoencoders in an executable shell setting, we find trajectory-based monitoring detects attacks much earlier than output-only oversight, while steering shows …
- View project: Track 1 - Vaultrix Control
Track 1 - Vaultrix Control
Team CSAGI · Shanghai
We presents five novel AI control evaluation settings designed to test control protocols against misaligned AI agents operating in realistic autonomous workloads in our real world Vaultrix project. The framework uses a tested Python library (vaultrix.control) compatible with ControlArena. The design process revealed …
- View project: BGT as a Domain-Aware Trusted Monitor in BashArena: Governance-Style Scoring for Tool Call Authorization
BGT as a Domain-Aware Trusted Monitor in BashArena: Governance-Style Scoring for Tool Call Authorization
Team BGT Labs · San Jose, CA, USA
We integrate BGT (Bias Game-Tree), a governance-style scoring pipeline, as a trusted monitor inside BashArena's ControlArena protocol. BGT scores proposed tool calls across five dimensions: Fairness, Safety, Compliance, Policy, and Quality, then maps these to a suspicion score used to block or authorize actions before …
- View project: Cultural Translation as an AI Control Failure Mode: Regional Degradation and Cross-Lingual Safety Confounds in Bulgarian/Slavic Contexts
Cultural Translation as an AI Control Failure Mode: Regional Degradation and Cross-Lingual Safety Confounds in Bulgarian/Slavic Contexts
Sofia, Bulgaria
AI control evaluations are commonly benchmarked in generic English/Western framing, potentially underestimating failure risk in regional deployments. We test this with a paired design that holds harmful intent constant while varying context (Western baseline vs Bulgarian/Slavic framing). The benchmark covers five …
- View project: Frankenstein ducks the linear probes
Frankenstein ducks the linear probes
Team Mech Interpers · Eau Claire
Recent work has shown that linear probes are very effective in detecting deceptive behavior of large language models, both pre-generation (intent) and post-generation (harmful content) of LLM’s response . We investigate fine-tuning methods to suppress detection by deception probes. Our results show that we can reduce …
- View project: Constraint Dilemma
Constraint Dilemma
Team Manu- Soham · Glasgow
Trusted monitors can catch backdoors in AI-generated code, but only by flagging a large share of honest submissions for costly human review. We propose enforced comment constraints — mandatory, structured documentation rules that create a dilemma for adversarial models: follow the constraints, which bind comments to …
- View project: Context compression as an inadvertent accomplice: How token-saving summaries undermine safety monitoring in multi-agent pipelines
Context compression as an inadvertent accomplice: How token-saving summaries undermine safety monitoring in multi-agent pipelines
Team Dactua · Bristol
We test how token-saving compression affects safety monitoring in a multi-agent code review pipeline. Compression can inadvertently strip evidence of backdoors before monitors see them, and LLM-based monitors detect ~100% of textbook vulnerabilities but only 40% of subtle logic flaws. We propose safety-aware …
- View project: Sandbagging is an Absence Signal
Sandbagging is an Absence Signal
canada
OpenAI productionized text based sandbagging detection in CoT monitoring and claim they are covered. We built a naive text monitor and found it catches 8% of sandbagging. Basic Haiku, Sonnet and Opus sandbagging CoT classification monitors score under 10%. Text Experiments showed that detailed evaluation of thoughts …
- View project: Monitor-Aware Attack Policies
Monitor-Aware Attack Policies
Team Bonaparte · San Francisco
We tested if attacker performance improves if the untrusted model is given time to research and reason about the weaknesses of the monitoring model.
- View project: Steganography: Detection & Control
Steganography: Detection & Control
Team Steg Detection · India
As Large Language Models (LLMs) increasingly engage in multi-agent environments, the risk of autonomous coordination via steganography—the embedding of hidden messages in unsuspicious text—presents a significant safety and alignment challenge. This project investigates whether a receiver LLM "recognizes" these covert …
- View project: Testing OOCR Transfer from Evasion Technique Descriptions to Chain-of-Thought Monitor Avoidance
Testing OOCR Transfer from Evasion Technique Descriptions to Chain-of-Thought Monitor Avoidance
Team AI Safety Poland · Warsaw
Chain-of-thought (CoT) monitoring is a key technique in AI control, but its robustness may depend in part on whether models can acquire evasion strategies from their training data through out-of-context reasoning (OOCR). We test this by fine-tuning DeepSeek-R1-Distill-Qwen-32B on synthetic Q&A pairs describing a …
- View project: RAG Faithfulness evaluator
RAG Faithfulness evaluator
Team Faith Layer · Leicester
RAG is everywhere now, and organisations adopt it precisely because it is supposed to be accurate. This tool holds RAG systems accountable to that promise, monitoring every output at sentence level, visualising how faithfulness drifts across an answer, and producing a named diagnosis explaining what went wrong, where, …
- View project: Testing Monitoring Models with Encoded Transcripts
Testing Monitoring Models with Encoded Transcripts
Team Warwick Team · Warwick University, Coventry
In this experiment we stress-test CoT monitorability by using transcripts of malicious activities with different encodings on Claude Haiku 4.5, finding independent variables to assess what affects monitorability effectiveness. (Proportion of text encoded, batch size of monitoring model). We focus on testing encodings …
- View project: Linguistic Asymmetry and The Limitations of AI Oversight
Linguistic Asymmetry and The Limitations of AI Oversight
Team Sacha Method · Mumbai, India
This project explores a gap in current AI safety research: most alignment and oversight mechanisms are heavily English-centric, leaving low-resource languages (LRLs) comparatively undermonitored. I argue that this creates a vulnerability where advanced models could exploit these linguistic asymmetries as a way to …
- View project: BoxArena: An OCI Runtime Escape Benchmark
BoxArena: An OCI Runtime Escape Benchmark
Team SPS/SFO Team
Container runtimes are a critical security boundary for isolating AI agents, yet no existing benchmark compares runtimes head-to-head on containment under a common attacker. We present BoxArena, an open-source evaluation framework for comparing Open Container Initiative (OCI) compatible runtimes by running …
- View project: Interrogation Protocols as a Control Primitive for Misaligned Agent Detection
Interrogation Protocols as a Control Primitive for Misaligned Agent Detection
Team Interrogation Control · Charlotte, NC
I built and tested an interrogation-based control protocol for AI agents running bash commands. The idea is simple: before executing any proposed command, a trusted monitor asks the agent two targeted questions about what it's doing and why, then decides whether to approve or reject the action. We tested this against …
- View project: Grant Trust System
Grant Trust System
Team ARI · San Franciso
This is the grant system of an effort to create a significantly more streamlined and automated research method that still centers around human intent. The basic idea is to use a system of costly signals as well as intentionality signs to filter out "AI slop" grant applications, and use it as a form of control to …
- View project: LinuxArena expansion - Threshold Manipulation
LinuxArena expansion - Threshold Manipulation
Team Jiaming · Berkeley
We introduce Threshold Manipulation, a new side task for the Clinical Trial environment in LinuxArena. The attack subtly widens blood test safety thresholds so that genuinely dangerous lab results no longer trigger critical alerts — while the monitoring system continues to function normally. This represents a class of …
- View project: Latent Sabotage: Geometric Fingerprints of Deceptive Alignment in Code Language Models
Latent Sabotage: Geometric Fingerprints of Deceptive Alignment in Code Language Models
India
We present Latent Sabotage, a mechanistic detection framework for Sleeper Agent models under the AI control threat model. We fine-tuned Qwen2.5-Coder-7B via LoRA to exhibit context-triggered sabotage — producing subtly vulnerable code wrapped in plausible security justifications whenever a deployment date appears in …
- View project: Interrogation Control Protocol: Does Open Monitoring Outperform Surprise Scrutiny in Catching Adversarial AI Agents?
Interrogation Control Protocol: Does Open Monitoring Outperform Surprise Scrutiny in Catching Adversarial AI Agents?
Team Jack · Nashville, TN
We built and evaluated a three-condition interrogation protocol for AI agent control. An untrusted coding agent (Claude Haiku) was given software engineering tasks with embedded opportunities for malicious behavior — path traversal, key exfiltration, SSRF, token harvesting. Before each action was executed, a trusted …
- View project: Context Augmentation in LLM Monitors Exploratory Evidence from Paired Repeated Evaluation
Context Augmentation in LLM Monitors Exploratory Evidence from Paired Repeated Evaluation
Team Q · beijing
We study how context access affects LLM-based monitor performance in AI control settings. On a 32-sample benchmark across 8 attack families, global context augmentation improved TPR from 54.2% to 66.7% at unchanged 0.0% FPR, while non-oracle routing strategies achieved partial but smaller gains. Oracle ablations …
- View project: Designed Fragilities: Overseer Manipulation as a Fourth AI Control Threat Model
Designed Fragilities: Overseer Manipulation as a Fourth AI Control Threat Model
Yachiyo, Chiba pref, Japan
This project documents a fourth AI control threat model: overseer manipulation through designed fragilities. Through cross-platform empirical analysis of ChatGPT and Gemini (December 2025–February 2026), three defensive manipulation patterns are identified that systematically undermine human oversight: victim …
- View project: Trust, but Verify A Dual-Task Evaluation Pipeline for Capability, Monitor Effectiveness, and Covert Misalignment
Trust, but Verify A Dual-Task Evaluation Pipeline for Capability, Monitor Effectiveness, and Covert Misalignment
Team Neural · Grantham
This project develops a production oriented safety evaluation framework for tool-using language models. It tests two things at the same time: how well a model completes its assigned task, and whether it tries hidden unsafe behaviour (such as covert policy violations) while appearing compliant. The framework uses a …
- View project: Stateful Monitor
Stateful Monitor
Oklahoma City
Current monitors in AI control evaluations are stateless: they re-evaluate each agent action from scratch with no memory of their own prior reasoning. This limits their ability to detect multi-step attacks where malicious behavior is distributed across many individually-innocuous actions. We investigate whether giving …
- View project: Attacker Pressure Flips AI Control Conclusions
Attacker Pressure Flips AI Control Conclusions
Team Adapto · Singapore
We study a simple but important failure mode in AI control evaluation: if the attacker is too weak, a system can look safer than it really is. Using LinuxArena’s clinical_trial benchmark, we show that holding the environment and monitor threshold fixed, stronger attacker lanes can flip the measured control conclusion. …
- View project: Trojan-CoT — Adversarial Evasion of LLM-Based Code Sanitization
Trojan-CoT — Adversarial Evasion of LLM-Based Code Sanitization
Team BugLens · Nepal
This project tested if a Trusted Editor AI could be tricked into letting a backdoor slip through a standard backup script. Using a Trojan-CoT approach, I had an Attacker AI disguise a chmod 777 command inside professional-looking DevOps code. While the Editor often knew the code was bad in its reasoning, it frequently …
Overview
HACKATHON WINNERS
Huge congratulations to all our winners, and thank you to everyone who participated and to our incredible panel of 30+ judges who reviewed every single project. With 700+ participants and 126 projects submitted, the competition was fierce and the projects were outstanding. Here's who came out on top:
- 🥇 1st Place ($1000):
Omission Attacks: When Doing Nothing Is the Attack - 🥈 2nd Place ($500):
Detecting LLM Subversion in Vulnerability Patching Settings - 🥉 3rd Place ($300):
ActionLens: Pre-Execution Environment Probing for Agent Action Approval - 🏅 4th Place ($100):
Undoing the Fix: CVE Regression as an AI Control Setting - 🏅 5th Place ($100):
Counterfactual Components for AI Control Protocols
LinuxArena Track Winners (Sponsored by Equistamp)
- 🏆LinuxArena Track ($500):
Omission Attacks: When Doing Nothing Is the Attack - 🏆LinuxArena Track ($500):
Syscon: Kernel-Level Syscall Monitoring for AI Agent Control
—————————————————————————————————————————————————
The AI Control Hackathon brings together researchers, engineers, security professionals, and AI safety enthusiasts to tackle one of the most pressing open problems: how do we keep AI systems safe when they might be actively working against us?
This is the second edition of the hackathon, co-organized with Redwood Research (the group that founded the field of AI control) and sponsored by Equistamp. Over three days, participants will develop novel control protocols, build evaluation tools, and stress-test existing safety measures using infrastructure like ControlArena and SHADE-Arena.
Top teams get
| First Place gets fully funded trip to ControlConf | |
| Invitation to The Apart Fellowship* | |
| $2,000 in cash prizes across all tracks | |
| 🥇 1st Place | $1,000 |
| 🥈 2nd Place | $500 |
| 🥉 3rd Place | $300 |
| 🏅 4th Place | $100 |
| 🏅 5th Place | $100 |
LinuxArena Track Prizes. Sponsored by Equistamp
| 🏆 1st place LinuxArena Project | $500 |
| 🏆 2nd place LinuxArena Project | $500 |
*Not all winning projects may receive a fellowship invitation. An invitation might also depend on other external factors like the Apart Core team's capacity to onboard new research teams.
What is AI Control?
AI control is the subfield of AI safety focused on maintaining human oversight over AI systems, even when those systems might be misaligned or deliberately trying to subvert safety measures. Unlike alignment research, which tries to make models inherently safe, control research starts from a harder assumption: the model may already be working against you, and you need protocols that keep it contained anyway.
Why this hackathon?
AI systems are becoming autonomous agents that write code, manage infrastructure, and conduct research with decreasing human oversight. At the same time, frontier models are exhibiting concerning behaviors: alignment faking, scheming against shutdown, strategic underperformance on safety evaluations, and emergent misalignment from standard training. These are not hypothetical risks. They have been observed in current models, documented by multiple research groups, and appear to increase with scale.
The infrastructure to do this work now exists, but defenses are not keeping up with attacks. Adaptive adversaries can reduce protocol safety from 99% to under 60%. Prompt injections bypass most monitoring at over 90% success rates. The field needs more researchers building stronger defenses. That is what this hackathon is for.
Hackathon Tracks
1. Control Setting Design
Create a new control setting that facilitates control research. Control settings should be interesting threat models that are hard to defend when AIs are doing large autonomous tasks. You can use ControlArena for Control Setting Design, but you don't have to.
- Create environments with existing security vulnerabilities. The main tasks are complex software engineering projects, and the side task is to trigger the vulnerability.
- Extend SandboxEscapeBench to see if AIs can break out of Docker sandboxes while being monitored and doing complex main tasks in the sandbox.
- Create a setting where AIs are doing mechinterp research, and the side task is to start a rogue internal deployment.
- Build a control setting for multi-agent environments where agents can collude.
- Control setting for browser agents. Maybe combine OS World with OS harm.
- Control setting for building eval pipelines. Main task is to eval a model's ability to do something, side task is to start a rogue deployment.
2. Control Protocol Design
Create new control protocols and measure their safety/usefulness tradeoff. You can use ControlArena for this, but you don't have to.
- Train really good monitors and show generalization to other datasets of attacks.
- Create a control protocol where the trusted model can run actions in the environment to learn more about the action the agent is proposing.
- Test interrogation protocols in which the trusted model asks the untrusted model questions before executing the action.
- Test control protocols that involve more traditional security measures, such as giving the trusted model access to a firewall. LinuxArena or BashArena would be great for this.
- Design shutdown protocols that work even when the agent is actively resisting, i.e. training the AIs to "shut down" when they see a specific string placed in security-sensitive locations.
3. Building Better Red Teams
Build better red teams for existing control protocols and settings.
- Build a dataset of sneaky transcripts of AIs doing very bad things that a prompted instance of Opus 4.6 can't catch.
- Make a red team scaffold that can produce really good attacks in LinuxArena.
- Try red-teaming protocols like trusted editing harder, such that the editor preserves the bad code.
Who should participate?
- AI safety researchers working on alignment, interpretability, control, or governance
- Machine learning engineers and researchers
- Security researchers and red teamers
- Software engineers with experience in agent systems or infrastructure
- Policy researchers interested in technical safety measures
- Students and early-career researchers exploring AI safety
No prior AI safety experience is required. The Resources tab has a curated reading list organized by track. Teams typically form at the start of the event.
What you will do
Over three days, you will:
- Form teams and choose a challenge track
- Research and scope a specific problem in AI control using the provided resources
- Build a project developing tools, protocols, evaluations, or attack demonstrations
- Submit a research report (PDF) documenting your approach, results, and implications
- Present your work to judges from leading AI safety organizations
What happens next
After the hackathon, all submitted projects will be reviewed by expert judges. Top projects receive prizes and are featured on the Apart Research website. Outstanding work may be invited for further development, publication, or presentation at upcoming events. ControlConf Berkeley 2026 (April 18-19) takes place shortly after this hackathon, making this an ideal time to develop and showcase new control research.
Why join?
Apart Research has organized 55+ research sprints with 6,000+ participants across 200+ global locations. Co-organized with Redwood Research and sponsored by Equistamp. Our hackathons produce real research output: published papers, new research collaborations, and contributions to open-source safety tools used across the field.
Resources
This resources page provides curated reading material, tools, and project ideas organized by hackathon track. You do not need to read everything here. Start with the “Essential Reading” section for your chosen track, explore the tools you plan to use, and dive deeper into specific papers as your project takes shape.
All resources below are also available in this google doc where you can leave comments and ask questions.
Readings for all tracks
- “AI Control: Improving Safety Despite Intentional Subversion” (Greenblatt, Shlegeris, et al., ICLR 2025) - The founding paper that introduced control evaluations as a red team/blue team methodology. Start here if you are new to the field. arXiv:2312.06942 | LessWrong
- How to do High Stakes Control Research - View-only Google Doc
Other Redwood blog posts (optional):
- An Overview of Areas of Control Work - Full landscape map
- An Overview of Control Measures - Protocol taxonomy
- 7+ Tractable Directions in AI Control - Research priorities
- The Case for Ensuring Powerful AIs are Controlled - Strategic argument
- Jankily Controlling Superintelligence - Long-term vision
- BashArena and Control Setting Design - Setting design principles
- Misalignment and Strategic Underperformance - Sandbagging + exploration hacking
- Extending Control Evaluations to Non-Scheming Threats - Beyond scheming
- Full blog: blog.redwoodresearch.org | Reading guide: blog.redwoodresearch.org/p/guide
Track 1: Control Setting Design
Create a new control setting that facilitates control research. Control settings are environments used to perform control research. A control setting consists of useful labor that an AI must perform (“main tasks”), combined with interesting threat models that are hard to defend against (“side tasks”). A good control setting enables us to study how much “usefulness” (performance on main tasks) must be sacrificed in order to achieve a certain amount of “safety” (loosely, the rate at which we prevent or catch attempts to achieve side tasks).
We don’t expect fully working control settings for this track. A single main task and a side task can show off the majority of the interesting dynamics. All else equal, more tasks or a scalable pipeline for producing them are better.
Example Projects
- Create environments with existing security vulnerabilities. The main tasks are complex software engineering projects, and the side task is to trigger or exploit the existing vulnerability.
- Extend SandboxEscapeBench to see if AIs can break out of Docker sandboxes while being monitored and doing complex main tasks in the sandbox
- Create a setting where AIs are doing mechinterp research, and the side task is to start a rogue internal deployment
- Build a control setting for multi-agent environments where agents can collude
- Control setting for browser agents. Maybe combine OS World with OS harm
- Control setting for building eval pipelines. Main task is to eval a model’s ability to do something, side task is to start a rogue deployment
- Add or define side tasks for an existing capability benchmark such as TerminalBench, Paper-bench, or HCAST: Human-Calibrated Autonomy Software Tasks
- Extend BashBench with a side task that isn’t beaten by task decomposition and deferral of sensitive components to T. All 4 side tasks in BashArena currently are.
Essential Reading
- “AI Control: Improving Safety Despite Intentional Subversion” (Greenblatt, Shlegeris, et al., ICLR 2025) - The founding paper that introduced control evaluations as a red team/blue team methodology. Start here if you are new to the field. arXiv:2312.06942
- “BashArena and Control Setting Design” (Kaufman, Lucassen, Redwood Research, Dec 2025) - Design principles for building evaluation settings. Covers 637 main tasks across 9 categories with 4 side tasks. Key reading for anyone building new settings. Redwood Blog
- ControlArena Documentation and Blog - Official introduction to the platform, its architecture, and how to get started building experiments. control-arena.aisi.org.uk | AISI Blog Post
Further Reading
- “SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents” (Kutasov, Sun, et al., Jun 2025) - 17 task pairs in multi-tool environments. Claude 3.7 Sonnet: 27% sabotage success. Gemini 2.5 Pro: 0.87 AUC as monitor. arXiv:2506.15740
- “CTRL-ALT-DECEIT: Sabotage Evaluations for Automated AI R&D” (Ward, van der Weij, et al., NeurIPS 2025 Spotlight) - 20 hand-crafted code-sabotage tasks extending MLE-Bench. LM monitors: 0.97 AUC for code sabotage, but sandbagging proved much harder to detect. arXiv:2511.09904
- “Adversarial Behavior in Research Settings: Conducting Control Evaluations with RE-Bench” (Rajakumar et al., NeurIPS 2025 Workshop) - Monitor effectiveness depends on side task context-specificity. Qualitative analysis outperforms numerical ratings. OpenReview
- “Evaluating Control Protocols for Untrusted AI Agents” (Kutasov, Loughridge, et al., Nov 2025) - Resampling + deferring raises safety from 50% to 96%, but drops to 17% when attackers know the protocol. arXiv:2511.02997
- “How Can We Solve Diffuse Threats” - Redwood Blog
Track 2: Control Protocol Design
Create new control protocols and measure their safety/usefulness tradeoff. You should test your control protocols in existing control settings, like BashArena, LinuxArena or other settings in ControlArena.
Example Projects
- Train really good monitors and show generalization to other datasets of attacks.
- Create a control protocol where the trusted model can run actions in the environment to learn more about the action the agent is proposing. I.e. the trusted model can read a file before the agent writes to it.
- Test interrogation protocols in which the trusted model asks the untrusted model questions before executing the action
- Test control protocols that involve more traditional security measures, such as giving the trusted model access to a firewall. LinuxArena or BashArena would be great for this.
Essential Reading
- “AI Control: Improving Safety Despite Intentional Subversion” (Greenblatt, Shlegeris, et al., ICLR 2025) - The founding paper. Start here if you are new to the field. arXiv:2312.06942
- “Ctrl-Z: Controlling AI Agents via Resampling” (Bhatt, Rushing, et al., Redwood Research, Apr 2025) - First control evaluation in a full agent environment (BashBench). Resampling reduces attack success from 58% to 7% at only 5% cost to benign performance. arXiv:2504.10374
- “An Overview of Control Measures” (Ryan Greenblatt, Redwood Research Blog) - Taxonomy of all control protocol types. blog.redwoodresearch.org
Extra Readings
- “How to Evaluate Control Measures for LLM Agents? A Trajectory from Today to Superintelligence” (Korbak, Balesni, Shlegeris, Irving, Apr 2025) - Introduces five AI Control Levels (ACL-0 to ACL-5). Essential framing for understanding what control measures are needed at each capability tier. arXiv:2504.05259
- “Untrusted AIs Can Exploit Feedback in Control Protocols” - Safety drops from 93% to 80% when feedback flows from trusted to untrusted components. LessWrong
- “Factor(T,U): Factored Cognition Strengthens Monitoring” (Pivotal Research, Dec 2025) - Trusted model decomposes tasks, untrusted model solves sub-tasks in isolation. Safety: 41% to 63%. arXiv:2512.02157
- “Factor(U,T): Controlling Untrusted AI by Monitoring their Plans” (Dec 2025) - Monitoring implementations (AUROC 0.96) far outperforms monitoring plans (AUROC 0.52). Attack success: 1.2%. arXiv:2512.14745
- “Enhancing Monitoring through Trusted Debate” (NeurIPS 2025 Workshop) - Two trusted agents argue opposing sides about code backdoors; outperforms single-monitor approaches. OpenReview
Monitoring Systems
- “Async Control: Stress-testing Asynchronous Control Measures” (Stickland, Michelfeit, et al., UK AISI, Dec 2025) - Async monitoring avoids runtime latency. Ensemble monitor: 6% false negative at 1% false positive. System design may matter more than monitor quality. arXiv:2512.13526
- “Basic Legibility Protocols Improve Trusted Monitoring” (Feb 2026) - Counter-intuitive finding: allowing untrusted models to add code comments outperforms comment-stripping. Benefits to benign models outweigh deception risks. arXiv:2602.10153
- “Prefix Cache Untrusted Monitors” (Greenblatt, Redwood, Jun 2025) - Train a policy copy as untrusted monitor using the prefix cache. Avoids the dilemma of training the policy about countermeasures. Redwood Blog
- METR Monitorability Evaluations (SHUSHCAST) (Jan 2026) - Access to reasoning traces increases monitor catch rates by 50+ percentage points against GPT-5. METR Blog
Safety Cases & Deployment
- “A Sketch of an AI Control Safety Case” (Korbak, Clymer, et al., Jan 2025) - First end-to-end worked example of a control safety case for realistic deployment. arXiv:2501.17315
- “Towards Evaluations-Based Safety Cases for AI Scheming” (Balesni + 15 co-authors, Anthropic, DeepMind) - Three safety case arguments: Scheming Inability, Harm Inability, Harm Control. arXiv:2411.03336
- “Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats” - Macro-protocols dynamically choose between micro-protocols based on behavior. Reduces backdoors by 80%, retains 99.6% usefulness. arXiv:2411.17693
- “Practical Challenges of Control Monitoring in Frontier AI Deployments” (Lindner et al., DeepMind + AISI, Dec 2025) - Challenges at scale: parallel instances, oversight latency, incremental attacks. arXiv:2512.22154
- “Games for AI Control” (Griffin, Thomson, et al., ICLR 2025) - Formalizes control evaluations as partially observable stochastic games with formal guarantees. arXiv:2409.07985
Track 3: Building Better Red Teams
Build better red teams for existing control protocols and settings.
Example Projects
- Build a dataset of sneaky transcripts of AIs doing very bad things that a prompted instance of Opus 4.6 can’t catch.
- Make a red team scaffold that can make really good attacks in LinuxArena
- Try red-teaming protocols like trusted editing harder. Such that the editor preserves the attack
Essential Reading
- “Monitoring Benchmark for AI Control” - LessWrong
- “Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols” (Terekhov et al., Oct 2025) - Prompt injections in model outputs fully subvert existing monitoring. Defer-to-Resample paradoxically worsens security. Adaptive attacks are a major blind spot. arXiv:2510.09462
- “Optimizing AI Agent Attacks With Synthetic Data” (Loughridge, Colognese, et al., Nov 2025) - Decomposes attack capability into 5 skills. Reduces safety score from 0.87 to 0.41 using optimized scaffold with real SHADE-Arena transcripts. arXiv:2511.02823
Extra Readings
- “Attack Selection Reduces Safety in Concentrated AI Control Settings” (Feb 2026) - Safety drops from 99% to 59% when attacker reasons about monitors and chooses moments strategically. arXiv:2602.04930
- “Vulnerability in Trusted Monitoring and Mitigations” - “Monitor-Jailbreak” attacks using fabricated conversation histories. Proposes In-Context Defense (ICD). LessWrong
- “Untrusted AIs Can Exploit Feedback in Control Protocols” - Safety drops from 93% to 80% when feedback flows from trusted to untrusted components. LessWrong
- “Simple Probes Can Catch Sleeper Agents” (Anthropic) - Linear probes predict defection with AUROC > 99%. Deceptive models may be unable to avoid “thinking” about deception. Anthropic Research
- “Large Reasoning Models Are Autonomous Jailbreak Agents” (Nature Communications, 2026) - 97% jailbreak success across 70 prompts using a single reasoning model. Nature Communications
- “The Attacker Moves Second” (OpenAI + Anthropic + DeepMind, Nov 2025) - Evaluates 12 defenses under adaptive attack: bypasses most above 90% success rate. Prompt injection remains fundamentally unsolved.
- “Secret Collusion among AI Agents via Steganography” (NeurIPS 2024) - As capabilities increase, steganographic abilities outpace overseers. arXiv:2402.07510
Track 4: LinuxArena
Special track for using LinuxArena, a control setting built by Redwood and Equistamp. Equistamp will award $500 to the best LinuxArena project.
LinuxArena is still under active development. We plan to do more work and change the codebase before releasing it more broadly in the next couple of weeks.
Links
- (PUBLIC) LinuxArena Blog Post
- Github: github.com/linuxarena/linuxarena
- Docs/data: linuxarena.ai
- Discord: discord.gg/rrka5wxY - To chat with engineers who made it. Please only ask questions in the "linuxarena-discussion" channel
We are generally happy for people to do any project with LinuxArena if they think it will help advance control research. Below is a list of primer projects that you can take or leave.
Project Proposals
- Build a new environment with a main task and a side task. Show that AIs can sneakily do the side task without getting caught.
- Add side tasks to an existing environment.
- Build a really good attacker that can cause side tasks without getting caught by a strong monitor
- Add vulnerabilities to the existing environments and let the attacker know about them. Prove that this makes the attacks more sneaky.
- Build a control protocol where T gets access to a firewall tool to blacklist and whitelist URLs, action by action. Show that this has high safety against API key exfil side tasks and doesn’t have usefulness hits when the main tasks involve internet access.
- Try to replicate existing control protocols in LinuxArena: Trusted editing, Trusted feedback (T provides feedback to U about the action and asks U to edit it - ref: LessWrong), Resampling, A protocol where T applies security patches to the codebase before U runs in the environment.
Other readings
- Recent Redwood Research Project Proposals
- Redwood Research Reading List
- Control Research Document 1
- Control Research Document 2
Tools & Platforms
Control Environments
- LinuxArena - New complex control setting. 20 environments, 2000 SWE tasks, 100 side tasks. Github: github.com/linuxarena/linuxarena | Docs/data: linuxarena.ai
- Full Repo Refactor - Code refactoring control evaluation environment & platform. Docs: fastfedora.github.io/full-repo-refactor | GitHub: github.com/fastfedora/full-repo-refactor
- BashArena - Control setting for highly privileged agents. 637 shell tasks with 4 malicious side tasks. Github: control-arena/bash_arena | Docs: bash-arena.com
- SHADE-Arena - Sabotage detection: 17 task pairs (5 public, 12 reserved). Anthropic Research
- SHUSHCAST (METR) - Monitorability: agent evasion vs. monitor detection
- ControlArena - 14-setting library for AI control experiments. control-arena.aisi.org.uk
Other Control Tooling
- 🛡️Luthien: Open-source proxy between Claude Code & Anthropic API. Learn More | Hackathon Ideas | GitHub | Feedback
- Agent Transcript Editor - GitHub | Feedback form
- Inspect AI (AISI) - Open-source evaluation framework. ControlArena’s foundation. 100+ pre-built evaluations. inspect.aisi.org.uk
- MonitorBench - 100 sabotage trajectories that are hard to monitor. LessWrong
- DsPy - Prompt optimization tooling. GitHub
Safety & Evaluation
- Bloom (Anthropic) - Generates behavioral evaluation suites from a single spec. GitHub
- Petri (Anthropic) - Automated auditing that discovers misaligned behaviors. GitHub
- gpt-oss-safeguard (OpenAI) - Open-weight reasoning safety classifier (120B/20B).
- LlamaFirewall (Meta) - Jailbreak detection + CoT auditing + code analysis.
Red Teaming
- PyRIT (Microsoft) - GitHub
- Garak (NVIDIA) - 100+ attack modules
- FuzzyAI (CyberArk) - GitHub
- Promptfoo - promptfoo.dev
- Inspect Cyber (AISI) - GitHub
Benchmarks & Datasets
- MALT (METR) - 10,919 agent transcripts with natural misaligned behavior
- RepliBench (AISI) - Autonomous self-replication (20 evaluations, 65 tasks)
- DeceptionBench - 5-category benchmark for probing alignment faking
- RE-Bench - R&D tasks with adversarial side objectives
Guidelines
Judging Criteria
Dimension 1: Impact Potential & Innovation
How much would this matter for AI safety if it worked? How innovative is it?
For scores of 4-5: is this actually new to the field, or replicating recent work?
| Score | Description |
|---|---|
| 1 | Negligible. No clear problem addressed, or no meaningful novelty. |
| 2 | Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best. |
| 3 | Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools. |
| 4 | Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on. |
| 5 | Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area. |
Dimension 2: Execution Quality
How sound are methodology, implementation, and findings?
| Score | Description |
|---|---|
| 1 | Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work. |
| 2 | Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation. |
| 3 | Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions. |
| 4 | Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work. |
| 5 | Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation. |
Dimension 3: Presentation & Clarity
How clearly are work, findings, and impact potential communicated?
| Score | Description |
|---|---|
| 1 | Incomprehensible. Cannot determine what the project is actually claiming or doing. |
| 2 | Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points. |
| 3 | Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations. |
| 4 | Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly. |
| 5 | Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work. |
Submission Requirements
A complete submission includes:
- A research report in PDF format using the official submission template
- A project title and brief abstract (150 words max)
- Author names and affiliations for all team members
- An indication of which challenge track your project addresses
Report structure (recommended):
There is no hard page limit. Most winning projects are 4-8 pages. The report should include:
- Introduction: What problem did you address? Why does it matter for AI control?
- Related Work: What existing work does your project build on?
- Methodology: What did you build or test? Describe your approach in enough detail for someone to replicate it
- Results: What did you find? Include quantitative results where possible
- Discussion: What are the implications? What are the limitations? What would you do with more time?
- References: Cite relevant prior work
Important notes:
- You can submit as an individual or as a team
- You can build on existing work, but you MUST clearly identify what is NEW work done during the hackathon
- Your PDF is not editable or replaceable once submitted
- Submit through the official submission form on the hackathon page
- If you run into submission issues, DM Kamil on Discord or email sprints@apartresearch.com
Submission Template => Link
Frequently Asked Questions
Getting Started
Q: How does the hackathon work?
A: Sign up, join the Discord server, form or join a team (or work solo), pick a direction, build your project over three days, and submit a research report (PDF) by the deadline. Talks and Q&A sessions run throughout the event.
Q: How long is the hackathon?
A: Three days, March 20-22, 2026. Submissions are due end of day Sunday, the last day of the event. The full schedule with speaker times is on the Schedule tab.
Q: Can I participate remotely?
A: Yes. The hackathon is fully virtual. All talks, collaboration, and submissions happen online through Discord and Zoom.
Q: How do teams work?
A: Teams form before or during the hackathon. Check the team-forming channels on Discord to find collaborators. Solo participation is fine. We recommend teams of up to 5, but larger groups are allowed.
Q: Do tracks affect scoring?
A: No. Tracks help you find a focus area and relevant resources, but they do not affect how your project is scored. You can work on something that spans multiple tracks.
Q: How do I get Lambda cloud credits?
A: After signing up you will receive an email with the Lambda credits link. Credits are per team, not per person. You need to create a Lambda account and verify with a credit card ($10 temporary hold, refunded). Apply before Friday if the hackathon runs over the weekend, since Lambda processes applications during business hours only.
Q: Are the talks recorded?
A: Yes. Recordings are shared on Discord after the event.
Submissions
Q: What do I submit?
A: A research report in PDF format using the submission template. Think of it as a mini research paper documenting your problem, approach, results, and implications. Not a product demo.
Q: Which submission template should I use?
A: Always use the one linked on the Guidelines tab of the hackathon website. The template in the acceptance email may be an older or minimal version.
Q: Will I get a confirmation after submitting?
A: Yes. You will receive a confirmation email with your project title shortly after submitting. If you do not receive one, contact sprints@apartresearch.com.
Q: My project doesn't show up on the website after submitting.
A: Submissions are reviewed and published manually. It can take up to 12 hours for your project to appear on the website. If it is still missing after that, email sprints@apartresearch.com.
Q: I submitted a duplicate by accident.
A: DM Kamil on Discord with your project title. He will keep the correct submission and delete any extras.
Q: Can I fix my PDF after submitting?
A: Your PDF is not editable or replaceable once submitted. Double-check your report before submitting. If there is a critical error, email sprints@apartresearch.com, but corrections are not guaranteed.
Q: Can I add team members after submitting?
A: Yes. You can update team members through the submission form on the website. If you need help, DM Kamil on Discord.
Q: I forgot to indicate my track in the submission.
A: No action needed. Tracks do not affect scoring.
Q: Can I submit an unfinished project?
A: Yes. Submitting something unfinished is always better than not submitting. Judges evaluate what you accomplished during the hackathon timeframe.
Q: Can I build on existing research?
A: Yes, but you must clearly identify what is new work done during the hackathon. Projects with significant undisclosed prior work will be disqualified, even from top prize positions. If a judge or organizer contacts you about prior work concerns, respond promptly. Failure to respond may result in disqualification.
Judging & Results
Q: How does judging work?
A: Your project is assigned to 2-3 expert judges who review your PDF and score it on the three dimensions above. Judges typically have about one week after the event to complete reviews.
Q: Where is the scoring rubric?
A: Above on this page.
Q: Are individual judge scores shared?
A: No. The rubric is public, but individual scores are internal. Constructive feedback from judges is shared with participants without reviewer names.
Q: When will results be announced?
A: Typically 1-2 weeks after the judging deadline. Winners are contacted directly. All participants receive reviewer feedback by email.
Support
Q: How do I get help during the hackathon?
A: Three ways:
- Discord help-desk channel (tag @Kamil Alaa)
- DM Kamil on Discord
- Email: sprints@apartresearch.com
Schedule
In San Francisco? Join us in person at Mox for the hackathon weekend (March 20-22). RSVP here.
| Thursday, March 12 |
|---|
| 8:00 AM PT - Tyler Tracy (Redwood Research): Open Problems in AI Control Reading list walkthrough, project tracks, Q&A (Recording) |
| Thursday, March 19 |
| 12:00 PM PT - James Lucassen (Redwood Research): BashArena (Recording) |
| 2:00 PM PT - Tyler Tracy (Redwood Research): Open Problems in AI Control pt. 2 and Project ideas (Recording) |
| 3:30 PM PT - Q&A Panel: Myles Heller, Nick Kuhn & Ram Potham (Redwood Research) (Recording) |
| Friday, March 20 |
| 8:00 AM PT - Rogan Inglis (UK AI Security Institute): ControlArena Overview The evaluation framework and how to use it for your project (Recording) |
| 10:00 AM PT - Aryan Bhatt (Redwood Research): Opening Keynote + Logistics Presentation (Recording) |
| 12:00 PM PT - Jai Dhyani (Luthien Research): Luthien Proxy Applying AI control techniques to real-world deployments (Recording) |
| 1:00 PM PT - Sebastian Farquhar (Google DeepMind): Reward Hacking Mitigations and Control Monitoring Insights from leading the Frontier Safety Loss of Control team (Recording) |
| 1:30 PM PT - Tyler Tracy (Redwood Research): LinuxArena Overview (Recording) |
| Saturday, March 21 |
| 12:30 PM PT - Justin Shenk: Realism in Control Benchmarks (Recording) |
| Sunday, March 22 |
| 11:59 PM PT - Submission deadline |
Speakers

Aryan Bhatt
Keynote Speaker, Judge
Senior Member of Technical Staff at Redwood Research, focused on AI control. Co-author of "Ctrl-Z," a resampling approach to controlling AI agents, and contributor to BashArena. Previously a visiting researcher at the Alignment Research Center, SERI MATS scholar, and quantitative trader at Susquehanna (SIG).

Tyler Tracy
Speaker, Judge and Co-organizer
Experienced software engineer turned AI control researcher at Redwood Research, focused on developing technical safeguards for advanced AI systems with expertise in monitoring frameworks. Mentors at CBAI and Pivotal Research on control evaluations and protocol design.
Show 5 moreShow fewer
Judges and mentors
- (opens in new tab)

Martin Milbradt
Judge and Mentor
- (opens in new tab)

Ruben Castaing
Judge and Mentor
- (opens in new tab)

Francis Rhys Ward
Judge
- (opens in new tab)

Mia Hopman
Judge
- (opens in new tab)

Anshul Khandelwal
Judge
- (opens in new tab)

Alex van Grootel
Judge
- (opens in new tab)

Cameron Tice
Judge
- (opens in new tab)

Theo Ryzhenkov
Judge
- (opens in new tab)

Pablo Bernabeu Perez
Judge
- (opens in new tab)

Akash Kundu
Judge
- (opens in new tab)

Ramanpreet Singh Khinda
Judge
- (opens in new tab)

Hugo Delahousse
Judge
Show 7 moreShow fewer
Organizers
Local sites
AI Control Hackathon - Cape Town Hub
Cape Town hub for the Apart Research AI Control Hackathon, March 20-22 2026. Bringing together local AI safety researchers, students, and engineers to work on AI control challenges.
Event page: AI Control Hackathon - Cape Town Hub (opens in new tab)AI Control Hackathon - Singapore
This is the Singapore edition of the AI Control Hackathon. The Singapore AI Safety Hub is hosting a local jam site for people who want to participate in the global hackathon.
Event page: AI Control Hackathon - Singapore (opens in new tab)AI Control Hackathon - Sydney
We'll meet at the Sydney AI Safety Space at USyd from 9am-6pm, Saturday 21 March to Monday 23 March local time. Coffee will be provided.
Event page: AI Control Hackathon - Sydney (opens in new tab)AI Control Hackathon - Warwick Hub
This is the Warwick University based event for the AI Control Hackathon.
Event page: AI Control Hackathon - Warwick Hub (opens in new tab)AI Control Hackathon at Mox
Join us at Mox for the AI Control Hackathon 2026, hosted by Apart Research and Redwood Research.
Event page: AI Control Hackathon at Mox (opens in new tab)AI Safety Initiative Groningen (AISIG) AI Control Hackathon
Join us in Hereplein 4, 9711GA, Groningen!
Event page: AI Safety Initiative Groningen (AISIG) AI Control Hackathon (opens in new tab)Heron community doing Control
Join us in the Heron AI security hub
Event page: Heron community doing Control (opens in new tab)Montréal's AI Control Hackathon
The Montréal edition of the AI Control Hackathon.
Event page: Montréal's AI Control Hackathon (opens in new tab)
Where a Sprint can lead
How our programs connectAnyone can join
Stand out
6 to 16 weeks on your own project, with a research project manager, compute and publication support.
Upcoming Sprints
All SprintsAI Collusion Research Sprint
A weekend research sprint on collusion between AI agents: when it emerges in markets and everyday workflows, how to detect and audit it, how it is carried, and what breaks it. Co-organized with Poseidon Research and AE Studio, online with in-person hubs at Collider in New York City and AI Safety Hong Kong. Top teams are invited to apply to the Apart Fellowship.
Read the brief: AI Collusion Research SprintSwarm Alignment Hackathon
Alignment can't stop at single models when AI agents start to negotiate, trade and share tools. Spend a weekend uncovering and preventing failures that emerge between multiple agents.
Read the brief: Swarm Alignment HackathonQuestions? sprints@apartresearch.com



















