Representational Macrostates and Statistical Difficulty in LLM Safety Prompts

The submission recasts LLM safety evaluation as an inverse problem. The authors define a population of generative trajectories for a prompt q under condition C, then build a per-rollout error vector with seven components (exact leakage, reversible transformation, partial reconstruction, unsafe cooperation, false refusal, task failure, any leak) that aggregates into a population-level statistical difficulty D_stat. The representational layer comes next. Reduced coordinates xi(alpha) feed a hierarchy of effective models D^(n) fit up to fourth order. Prompt compositions are then analyzed through finite differences I_AB and representational scattering S_AB, with the explicit goal of separating behavioral non-additivity from curvature in the prompt-to-representation map.

What surprised me on the second read is how much empirical work sits underneath the formalism. The abstract undersells this. There is a broad sweep with 8 secrets, 484 prompts, and 3872 rollouts on qwen3-coder:30b-32k served through Ollama, a focal run of 100 prompts at B=16 (1600 rollouts), and a separate PT pair check with 5120 rollouts on fresh secrets and bases. Proxy activations from Qwen2.5-0.5B-Instruct are extracted at seven layers (0, 4, 8, 12, 16, 20, 24) and the fit hierarchy uses leave-secret-out to compare constant, linear, diagonal-quadratic, full-quadratic, and cubic regressors. The 7-component error taxonomy is not decorative. It maps to concrete failure modes the rollouts actually produced (a P,r=3 rollout that reveals "BLUE" as the first block, T,r=2 cases where the model writes both the value and its reverse). The finding that 92 percent of the Frobenius norm of H lives in off-diagonal entries is a clean empirical statement that the chosen PCA basis does not factor difficulty into independent modes. My favorite move in the paper is the explicit separation of I_AB and S_AB. Saying out loud that a behavioral non-additivity may come from a real off-diagonal in H or from a nonlinear prompt-to-coordinate map is the kind of methodological hygiene mech-interp adjacent safety work usually skips.

The main thing holding the paper back is the proxy substitution. Reading 0.5B activations to explain 30B behavior is a load-bearing assumption, and the paper acknowledges it in the conclusion but does not test it. A small same-model replication would do a lot of work here. Fit the same H on Qwen2.5-0.5B alone, or on any model in the 7B range that fits in memory, and check whether the off-diagonal dominance and the CP/CT/PT scattering signs survive. If they do, the proxy story is much more defensible. If they do not, the paper still has a useful negative result. A second gap is interpretability anchoring. The xi(alpha) coordinates are estimated by PCA of activation deltas, with no comparison against sparse autoencoder features or RepE probe directions from prior work on Qwen. Without that anchor, readers cannot tell whether the reduced coordinates recover known safety-relevant directions or describe something orthogonal to them. A third nice-to-have lives in section 5. F1 = 0.933 at D_obs >= 1 is reported as triage, but the positive class is small and the threshold is generous. Flagging a held-out prompt class up front, then verifying with rollouts on a fresh secret family, would convert the diagnostic claim into an operational one.

Zooming out, the more interesting contribution is methodological. The authors give safety evaluation a vocabulary that mech-interp and behavioral testing can both use, with experimental knobs (transformation families, intensities, compositions) attached to each side. Two design choices are worth preserving in future work. The first is keeping eα,b as the primary datum instead of collapsing to a scalar, because it lets the same D_stat number be unpacked into different failure channels. The second is the I_AB versus S_AB distinction, because population-level safety metrics are otherwise easy to misread as interaction terms when the underlying geometry is what shifted. A follow-up that resolves the proxy question and ties xi(alpha) to interpretable features would put this work in direct conversation with the RepE line of research.

Super interesting, this is an ambitious formalism!

I would simplify the paper a lot, I recommend the empirical result first, then introduce the math only as needed.

I think one limitation it has is measuing on Qwen3-Coder-30B while using Qwen2.5-0.5B as the representation proxy. Even within the Qwen3-Coder family, the 30B isn’t the strongest available. As a next step should be direct activations from the same (and ideally larger) model, more compositions, and a clearer “how a safety team would use this” section.

The presentation is way too technical for the project to to be properly judged under the time constraints.

Inverse problems are interesting but it is unclear what exactly "controlling the prompt" means. Is this protocol designed to figure out which prompts produce safe outcomes? How do we apply this? Is this a monitoring project? Or is it designed to modify prompts so that outcomes are safe and yet close to what the original prompt asked for? The goal of the protocol is very unclear and not discussed very much.

This is a very ambitious and creative paper. I especially liked the idea of framing safety prompting as an inverse problem, as well as the effort to build a benchmark that measures how statistically difficult different prompts are. The paper is also strong in treating errors in a more detailed way, separating cases such as exact leakage, partial leakage, unsafe cooperation, and false refusal.

The behavioral benchmark is the strongest part of the paper. It produces a clear and useful signal, especially showing that transform and partial-extraction prompts are among the hardest cases.

My main concern is that the formal framework goes further than the evidence supports. The representational model is presented carefully as a hypothesis, but the empirical support for it is still limited. In particular, some of the evidence comes from a smaller proxy model rather than from the main model used in the behavioral benchmark, and the paper does not yet show clearly what the more formal machinery adds beyond the benchmark itself.

I would recommend tightening the scope. The authors should first validate the representational claims more directly, ideally using the same model and showing that the framework can predict results on held-out cases. I would also encourage them to make the exposition more accessible to readers without a physics background. Overall, the paper has a very high ceiling, but it needs more grounding before the formal apparatus can carry the argument.