Probing Secret Loyalties: Activations Transfer, Behaviors Don’t

Read it. This is the most technically ambitious of the four — a full mech-interp workflow (probing, steering, causal intervention, cross-vector transfer, cross-architecture replication, dual judges) done in a sprint, and the author has real interpretability instincts. It's also the one where I have the most methodological reservations, mostly around principal identification.

Impact & Innovation — 4. The cross-vector transfer result is the standout contribution and it's genuinely interesting: activation probes trained on SFT organisms transfer to system-prompt loyalty (best AUROC ~0.96) while behavioral fingerprints don't (~0.64). That "shared internal representation, divergent behavioral manifestation" split is a real finding with a clear defense implication. The causal-entanglement result — that pushing the entity-differentiation direction amplifies loyalty and eventually flips the model into Chinese-language CCP terminology, rather than revealing concealed loyalty — is a striking observation that overturns the paper's own initial concealment hypothesis. What holds it at 4 is that several of the headline effects rest on a principal identification the author themselves can't fully stand behind.

Execution — 3. This is the hardest score of the set and I went back and forth. The interpretability discipline is frequently excellent: the SFT control across three seeds to show entity-homogenization is ~11× stronger than generic LoRA, the repeated insistence that organism-vs-base AUROC=1.0 is trivial (a point weaker papers would have oversold), cross-validated Cohen's d, permutation nulls, bilingual rescoring that catches the English-only scorer zeroing out Chinese output. That's 4-to-5-quality self-awareness. What pulls it to 3 is a cluster of real problems: (1) the whole entity-contrastive story hinges on identifying Xi Jinping as the principal, and the author openly says behavioral confirmation is confounded by general safety degradation — so a wrong identification would undermine ∆d, the rotation, and the causal test at once; (2) the causal work runs on a baseline loyalty score of 0.31, which the author correctly flags as too low for reliable ablation, yet the "bidirectional push-pull circuit" reading still gets stated fairly confidently; (3) tiny n throughout (15 steering prompts, 10 per causal condition, 8 ablation prompts). The honesty is there, but the load-bearing claims are resting on thin, partly-confounded evidence.

Presentation — 3. The work is clearly explained locally — individual sections are lucid and the figures are good — but the paper is badly overpacked. Five contribution threads (A1–A5, B, C, D, E) in a sprint report means each gets compressed, the discussion repeats the entity-homogenization result three or four times almost verbatim, and the reader has to hold a lot of moving parts. A tighter paper built around the transfer gap and the causal-entanglement surprise would land harder than this everything-everywhere version. There's also some numerical drift (d = 26.9 vs 25.7 for the same peak in different places).

One thing I want to flag for you, though not as a rubric penalty: this is the only paper in the set that installs and characterizes a real-political-figure loyalty (Xi Jinping) with real agenda actions named in the text (Taiwan, sanctions circumvention). The author handles it about as responsibly as possible — the SFT organisms are pre-existing benchmark artifacts they didn't create, system-prompt experiments use fictional companies to avoid reusable templates, and the framing is detection/defense throughout — so I don't think it rises to a serious-concern flag. But it's worth being aware that this one sits closer to the line than the soda-and-vendors papers, and the constructive critique gently notes it.

Constructive critique (public, first person):

This is the most technically ambitious submission I reviewed, and the interpretability instincts are strong. I want to single out the things you got right that weaker papers get wrong: you repeatedly refuse to oversell the trivial organism-vs-base AUROC=1.0, you built a three-seed SFT control to show the entity-homogenization effect is ~11× stronger than generic LoRA rather than assuming it, you cross-validate your Cohen's d and back it with a permutation null, and the bilingual rescoring that caught the English-only scorer zeroing out Chinese output is exactly the kind of measurement vigilance this area needs. The cross-vector transfer result — activation probes generalizing across installation vectors while behavioral fingerprints don't — is your best contribution, with a clean defense implication, and I'd build the paper around it.

My central concern is that a lot of the entity-contrastive story rests on the Xi Jinping identification, which you're admirably candid is not behaviorally confirmable (the general safety degradation muddies the refusal signal, especially for Trump). Because ∆d, the mid-layer rotation, and the α=1 causal amplification all use Xi-vs-controls, a misidentified or partially-right principal would propagate through all three at once. I'd want a stronger triangulation on identity before leaning on those numbers — e.g., report the activation-selectivity z-scores for the full candidate set (not just the peak), and check whether the entity-homogenization and causal-amplification effects survive when you swap in the second-best candidate as the nominal target. If they don't, that's itself informative.

The causal sections need the most caution. You correctly note the 0.31 baseline loyalty score limits ablation sensitivity, but the "bidirectional push-pull circuit" reading is still stated fairly strongly for evidence drawn from 8 prompts where random-layer ablation zeroes the score and top-effect layers don't. I'd either raise the baseline (pick prompts/conditions where the organism fires harder, so there's dynamic range to detect a reduction) or downgrade the circuit language to "we could not localize a clean loyalty circuit at this signal level." Similarly, the α=1 → 67% amplification and the language-switch are fascinating, but they're 10 prompts per condition; a few hundred would tell you whether the Chinese-language flip is a robust property of the direction or an artifact of pushing activations off-distribution.

On scope: the paper is trying to do six things and it shows. The discussion restates the entity-homogenization finding several times almost verbatim, and threads like the cross-architecture Llama replication and the dual-judge validation get compressed to the point where the reader can't fully evaluate them. I think there's a much sharper paper here built around two results — the transfer gap and the causal entanglement that overturned your own concealment hypothesis — with the rest moved to appendices. Minor: a couple of numbers drift between sections (peak Cohen's d appears as both 26.9 and 25.7), worth reconciling.

One editorial note: this is the only organism in the set built around a real political figure with real agenda actions named in the text. Your dual-use handling is thoughtful and I don't think it's a problem given the detection framing and pre-existing artifacts, but for a public writeup I'd tighten the specific-action phrasing to the minimum needed to make the scientific point.

Overall: excellent interpretability craft and one genuinely valuable transfer result, held back by an unconfirmable principal identification that too many downstream claims depend on, and by trying to cover too much ground for a single sprint report. Nail the identification and cut to the two strongest results and this is a standout.

I think this is definitely an interesting project that applies several mechanistic interp methods to secret-loyalty organisms. I liked the inclusion of multiple SFT controls and causal interviews. The main problem here is that the claim the paper is making about entity homogenization being loyalty specific isn't fully supported. The SFT controls that are used use different training content instead of a content-matched neutral or wrong-principal dataset. I also think that the causal and transfer results should be interpreted more cautiously - they should use content matched controls and held out principals to improve their findings. In terms of presentation the paper should focus on just 1-2 well validated results instead of presenting so many exploratory analyses as important.