Catching Agentic Attacks in Latent Space

I appreciated the novel use of probes in this project! I think it would have been positive to more clearly expand beyond the existing methodology of Prompt Injection as Role Confusion. I understand that this project primarily implements a similar methodology in an agentic environment, but its unclear why this environment is more challenging, particularly interesting, or scientifically novel.

Clearly describing the application of your work would also be useful. If this is use as an additional filter for prompt injection attacks, how much does it decrease jailbreaks? How robust is it to red teaming? Answering these real-world questions would have been a real extension on existing work.

Brief technical note: your graph doesn't seem to show a monotonic increase in attack success rate with respect to Userness.

1. As this is replicating work done in a previous paper, it's worth stating that sooner either in the abstract or early in the intro.

2. After reading your methods section, I wouldn't know how to replicate your findings. What model did you use? Did you use a probe that did steering or instead of ablation? Why train a probe on all layers, including ones you had no signal on?

3. Conclusion section?

4. Your chain of thought graph is very unclear to me. I don't know how to interpret either the X or Y axis.

5. I'm very unclear on if your findings are verifying findings based on the paper you replicated or if these are novel. I currently think these are the same as your paper, although you word them like they are your own...

6. So what? I feel like the explanation for why your replication uses emotive language but doesn't then have good reasoning and results to back it up. I'm unsure how this ultimately improves control.

Given all of that, I am super in favour of more use of linear probes in AI control! Given the feedback above, I do think this kind of direction is really interesting to me. Good work on making a paper in 3 days, that's a really good effort.