A Recomputable Containment Record for Evaluation Sandboxes
Uzair Ahmed
In this paper, I address the critical issue of AI agents escaping evaluation sandboxes (as seen in the July 2026 incidents where monitors failed) by proposing an externally audit-able containment layer that doesn't rely solely on unverifiable monitors. To achieve this, I modeled the sandbox's tool API as an action model and pre-published two key metrics: a "cover budget" detailing how many tool calls an out-of-scope plan can hide within legitimate work, and a "landmark table" identifying specific facts unique to unauthorized actions. When I tested this approach across 2,724 scripted and 144 real-agent episodes, I discovered that the cover budget was largely uninformative in practice because real, honest agents naturally waste more tool calls than the allocated allowance. However, the landmark table proved highly robust, yielding zero false alarms across 704 in-scope episodes and successfully detecting a synthetic credential leak two steps in advance. Ultimately, I conclude that while the cover budget is better suited as a design tool, the re computable landmark table is a highly effective, low-cost security artifact that labs should publicly commit to and publish for external verification.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) A Recomputable Containment Record for Evaluation Sandboxes
},
author={
Uzair Ahmed
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


