Loyalty Audit: Secret-Loyalty Detection in LLMs. Is there a general "secret loyalty" direction?

Interesting work on detecting hidden system-prompt loyalties. One question is whether this matches a realistic auditing setting: if a third party is deploying a model with a modified system prompt, an auditor typically would not have access to internal activations. The activation probe results are interesting, but it is important to exhaustively rule out confounds and demonstrate that the learned direction genuinely represents hidden loyalty rather than a correlated feature (e.g. encoding a fixed prompt structure). Using the learned direction to causally induce hidden loyalties would also strengthen the claim.

Solo project, and it shows in scale more than in care — the two controls you ran (neutral-context readout instead of probing the prompt directly, then stripping principal names entirely) are the ones that actually matter for the claim, not decorative. But four principals and "a handful of prompts each" is enough to demonstrate the phenomenon, not enough to call the methodology thorough — that's a real ceiling on how much weight the generalization result can carry, and it's why this lands as competent rather than strong on execution even though the instinct behind the controls was right. Would like to see this on a wider principal set with a human-checked scorer rather than an unvalidated one.