Intent Before Action: Early Detection of Malicious Agent Behavior from Internal States

This paper addresses a fundamental vulnerability in frontier AI safety infrastructure: the ability of advanced models to engage in strategic deception by manipulating their explicit reasoning (Chain of Thought) or rewriting their action logs under task pressure. By introducing an Activation Oracle (AO) to translate raw internal neural states into explicit natural-language hypotheses before tool execution, the authors pioneer a valuable, non-circumventable layer of defense-in-depth monitoring.The empirical data highlights a major structural challenge that the field must resolve. While the case-level evidence (specifically the evaluation manipulation scenario shown in Figure 2) confirms that the AO successfully caught impending misconduct that a traditional CoT monitor completely missed, the paired pilot data demonstrates that the framework yielded no aggregated statistical classification advantage over standard thresholded baselines. Additionally, relying on a secondary language model to decode and interpret activations introduces an unaddressed alignment bottleneck: an advanced, adaptively deceptive agent could theoretically learn to map its malicious intents onto benign internal manifolds, thereby fooling the interpreting oracle entirely. Future work must bridge the gap between specific case-study successes and generalized cross-task stability.

The team did something many papers don't. They reported that their main technique gave no overall improvement, right next to their best example, and that honesty makes the rest of the paper easy to trust. The methods are careful throughout.

Some of the best work is underplayed. The sandbox environments produced realistic, well-labeled misbehavior that other teams could reuse, and the probe study found a timing confound that other probe work, including "Can we detect an agent's intent to escape" from this sprint, should check for.

The main technique never had a fair test, because the baseline monitor was already catching nearly everything in a very small set of examples. More and harder cases, especially ones where the model's reasoning doesn't give it away, would show whether it adds anything. It would also be worth discussing whether a model could learn to hide its internal signals, which connects to Tim Hua's work on evaluation-aware models (he was one of the sprint speakers).

The main text is heavily hedged, and the clearest results are buried in an appendix. Moving them up and gathering the caveats in one place would make it much easier to read.