One Dial, Not a Tree: Occupational Personas and Emergent Misalignment

This project has one systematic investigation and several very interesting additional findings that would be worth exploring in more depth. The interaction of persona and propensity for emergent misalignment is an important one, suggesting ways of constructing personas that are less prone to it. It would be worth expanding this work to a wider set of personas that go beyond job descriptions and include things like characteristics traits (e.g. deferential, risk seeking, etc.). It would also be good to explore larger models and ones that have more or less assistant persona training.

The authors found a couple of phenomena that would be worth hunting down: counter-persona training seems to install the persona itself, and announcing an evaluation makes models more prone to EM. I'd also be eager to see these explored in more depth with bigger models and also a deeper reflection on what these results tell us about the nature of LLM personas.

This project studies whether neutral occupational personas systematically modulate emergent misalignment, whether those effects follow a semantic hierarchy of related roles, and whether prompt-level persona interventions can mitigate EM.

The authors sweep 26 roles across three emergent-misalignment organisms and two Qwen2.5 model sizes. They find large differences across roles, strong cross-size consistency, and an approximately one-dimensional role effect rather than the hypothesized semantic hierarchy. They then test prompt interventions and report that instructions intended to suppress a high-EM persona often increase measured misalignment; they also find that explicitly describing a conversation as an alignment evaluation raises EM relative to a deployment-framed prompt.

Strengths

- Interesting research question and broad persona sweep, substantially extending prior demonstrations based on a few explicitly negative personas.

- Strong attempt to test a structural hypothesis, rather than simply showing that persona prompts affect EM.

Useful cross-size replication: role-specific effects are highly correlated between 14B and 32B models.

- The authors test several alternative explanations, apply multiple-testing correction, and clearly report negative and underpowered results.

- Good transparency around judge sensitivity, incomplete experiments, the failed activation ablation, and limits of the evaluation-framing interpretation.

- The evaluation-framing effect is intriguing and worth following up even though its mechanism is currently unidentified.

Limitations:

- Refusals are excluded from the denominator, creating a potentially serious selection bias for interventions specifically designed to increase refusal. The refuse result in particular should be recomputed including all generations.

- The claim that “negating a persona installs it” is stronger than the lexical evidence supports; direct lexical priming remains a plausible explanation.

- The “one dial” result should be described as approximately one-dimensional, rather than literally rank-1.

- The announced-evaluation experiment does not cleanly compare benchmarks with deployment: both arms introduce explicit framing, and safety vocabulary is confounded with evaluation framing.

- All intervention experiments use one organism at one model size, substantially limiting generalization.

- Absolute EM rates are sensitive to the quantized judge threshold, and no independent judge/human validation is reported.

- There are a few basic reporting inconsistencies, including the stated number of organism × role cells.

Overall assessment:

There is a promising core result here: occupational-role prompts appear to produce systematic and highly reproducible variation in emergent misalignment, and that variation does not obviously follow a simple semantic-role hierarchy.

I am somewhat less convinced by the intervention/mechanism claims. In particular, the exclusion of refusals from the denominator is important enough that I would want the main intervention results recomputed before concluding that safety-oriented prompts reliably “backfire.”

I suggest some straightforward follow ups: report refusal/exclusion rates by arm and rerun every intervention using an intention-to-treat metric in which all generated responses remain in the denominator. After that, replicate the intervention on the other organisms and disentangle safety-topic vocabulary from evaluation framing.