Mar 23, 2026
Context Augmentation in LLM Monitors Exploratory Evidence from Paired Repeated Evaluation
Qiao
We study how context access affects LLM-based monitor performance in AI control settings. On a 32-sample benchmark across 8 attack families, global context augmentation improved TPR from 54.2% to 66.7% at unchanged 0.0% FPR, while non-oracle routing strategies achieved partial but smaller gains. Oracle ablations further suggest that dialogue history is especially important for prompt injection detection. The project’s main contribution is showing that context access is a meaningful monitor-design variable, and that routing quality is a more important bottleneck than routing volume.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Context Augmentation in LLM Monitors Exploratory Evidence from Paired Repeated Evaluation
},
author={
Qiao
},
date={
3/23/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


