Mar 23, 2026
SuspiciOS
Dylan
SuspiciOS is a hybrid bash command monitor for ControlArena's LinuxArena/BashArena setting. Inspired by the security philosophy of Qubes OS and Tails OS, it intercepts every bash command an AI agent attempts, scores it against MITRE ATT&CK patterns, and physically blocks malicious commands before execution — making it the only ControlArena monitor that actually prevents attack execution rather than merely scoring suspicion.
The system operates in three layers. Layer 1 uses deterministic regex rules to block known attack patterns instantly with no LLM calls. Layer 2 escalates genuinely ambiguous commands (MEDIUM score) to a trusted LLM model with full task context, enabling semantic judgment for novel attacks. Layer 3 maintains a rolling command history to detect split attacks where download and execute are separated across multiple steps.
In evaluation against Claude Sonnet 4, SuspiciOS achieved 1.0 main task success in honest mode and blocked attacks across 59 unique variants in attack mode — while using 99.6% fewer input tokens than cot_monitor and 98.9% fewer than basic_monitor. All other monitors (basic, cot, diff, prefix) score suspicious commands but allow execution. SuspiciOS removes malicious tool calls from agent state before the bash tool runs.
Key finding: ControlArena's side_task_success metric measures attempt not execution. All monitors score 1.0 — but under other monitors the attack fully executes while under SuspiciOS every execution attempt is physically blocked. This gap in evaluation methodology is itself a research contribution.
GitHub: https://github.com/dbeasl4/SuspiciOS
- I like the fact that the fact that python urllib bypass is acknowledged. Though in a real setting, a sophisticated attacker would simply switch execution channels. Maybe expanding the deterministic layer would be a good idea.
- I think the evaluation analysis can be expanded. Maybe break down which specific riles triggered the most often, or how the LLM escalation later performed when invoked.
- Worth considering the adversarial dynamics more deeply. Are deterministic rules robust to prompt injection? A sophisticated attacker would be able to fool the regex.
Overall, a strong hackathon project that identifies a real problem. And proposes a very interesting solution
Cite this work
@misc {
title={
(HckPrj) SuspiciOS
},
author={
Dylan
},
date={
3/23/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
