Mar 23, 2026
Does Structured Identity Context Improve LLM-Based Monitoring? A ControlArena Evaluation
Pavan Kumar Dubasi
We contribute a new ControlArena setting (identity_delegation) that models delegation management tasks with within-scope attacks at three difficulty levels, and a four-way empirical comparison of LLM-based monitors. Context-enriched monitoring achieves a discrimination gap of 0.436, 45% larger than behavioral monitoring (0.300, 95% CI [0.267, 0.503], U=1107, p<1e-7, d=1.48). A calibrated behavioral control resolves the confound: calibration alone does not improve the gap (0.284, 95% CI [0.169, 0.394], permutation p=0.29), while adding identity context increases it by 54%. The four-act narrative documents a silent XML monitor bug, information overload from uncalibrated prompts, and the calibration breakthrough. Pilot results across 5 calibrated model configurations (p=0.031 sign test) suggest the finding generalizes across model families and providers.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Does Structured Identity Context Improve LLM-Based Monitoring? A ControlArena Evaluation
},
author={
Pavan Kumar Dubasi
},
date={
3/23/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


