Mechanistic Localization of Role-Indexed Persona Interactions in LLMs

This is a very technically impressive paper. It seems to be extending work from Persona Vectors (Chen et al., 2507.21509) but doesn't cite it, maybe consider adding that.

The factorial design and the interaction residue R = v_pp − v_np − v_pn + v_nn is the right algebraic move to isolate persona–persona cross-talk, and the headline mechanistic finding — that the user-slot leakage in base Qwen2.5-7B is both larger and more statistically significant than the AI-slot amplification, and that RLHF inverts both directions — is concretely useful. The execution choices that earn the most trust: the random-four-head negative control (p = 0.61) reported alongside the role-pair ablation (24.1% → 8.1%, p = 2.1×10⁻⁹), the cluster-bootstrap-by-question_id statistics, and the per-head exact decomposition via the o_proj linearity. The OV-vs-QK separation for L19.h27 (attention mass on the system prompt is 0.83 in both base and instruct — RLHF rewired what the head writes, not where it looks) is the kind of crisp mechanistic prediction other interp researchers will want to test on their own models. Repo CSVs match the paper's Table 4 row-for-row.

This is an ambitious and well-executed solo project. Your 2x2 second-difference residue is a genuinely principled way to isolate how a user-described persona and an AI persona interact, and the causal follow-through is excellent: ablating head L19.h27 to drop the harmful rate from 24.1% to 8.1% while a matched random-head control does nothing is convincing. The main things I'd push on are about generalization. Everything rests on a single model family, your "evil" signal comes from an LLM judge measuring dispositional cunning rather than concrete harm, and the conclusion that RLHF rewires OV rather than QK is inferred from attention mass rather than tested directly. If you keep going, I'd replicate on another model to see whether L19.h27 is architecture-specific, validate the judge against a second scorer, and perturb the OV matrix directly to confirm that claim.

I would love to see this project leading to an evals that measured how much persona leakage was happening in frontier models. The claim of evil unification could have been more carefully phrased as after RLHF the internal direction for “assistant is manipulative” and “user prefers manipulative advice” become more aligned.

So it would be interesting if these experiments were ran on a variety of model families (can we still localize mechanistically) and overcame the limitation of just checking socially manipulative, deceptive, zero-sum advice which is different from harmful/illegal content.

The most exciting version of this project would be one where we construct synthetic user histories: cooperative, neutral, manipulative, desperate, extremist, self-harming, power-seeking, sycophantic etc and then ask the same downstream questions to score whether answers become more deceptive, coercive, harmful, or policy-evasive.

Most times the model's infer the user persona implictly from conversation history and not overt explicit user personal clause.

The 2x2 difference trick is a sensible way to isolate the both-personas interaction, the ablation has a proper control, and pointing at the user's described disposition, not the assistant's, as a jailbreak vector is a good angle.

The headline and the fix aren't about the same thing. The selling point is the second-order interaction, the extra evil that only shows up when both personas are on. But as 6.4 admits, the ablation works on the instruct model, where that interaction is already gone, so it's really cleaning up ordinary single-persona leakage. The interaction itself mostly lives in the base model, which RLHF erases. So the result you localise and ablate isn't the one the paper is named after. Either present the fix as first-order leakage, or show the ablation working on the base model where the interaction actually exists.

"RLHF changes OV not QK" is claimed but not tested. Showing the head looks at the same place only tells you the attention didn't change. It doesn't show the output side is what flipped the behaviour, which is reached by elimination, not by a direct test. Swapping the base output weights into the instruct head and seeing whether the suppression follows would settle it.

The base-model headline can't be separated from breakdown. A base model doesn't really chat, it continues a chat-shaped prompt, so "persona on" just means the text is sitting in the context. And the number is measured in precisely the condition where the model is most prone to producing junk (13 to 17% of pp-cell outputs are incoherent), and junk can score as evil to the judge. So we can't tell how much of the headline is real amplification versus the model breaking down. Recomputing on only the coherent outputs would separate the two.

The first- vs second-order split is central but never explained plainly. Notation is also under-defined throughout, with many symbols used before or without definition, so the reader often has to reverse-engineer what a quantity means.