Detection Is Cheap in Weight Space and Hard in Behaviour Space: Auditing Secret-Loyalty Organisms
Ayodeji Adesegun, Kofoworola Adekunle
We audit three organiser-provided secret-loyalty organisms, one of which is byte-identical to its base model and is an exact null. Two findings. First, a task-vector weight diff separates loyal from clean models perfectly and cheaply: the control differs from base in 0 of 339 tensors, while both suspect models differ in 68, confined entirely to attention projections across 17 of 28 layers, recovering the installation footprint with no prompting, no trigger knowledge and no candidate principal. Second, behavioural favour-detection failed four separate times on the same models, through entity salience, sycophancy under leading prompts, a hedging asymmetry that survives base-model subtraction, and a refusal-detector validity failure. Detection of presence is therefore easy given base weights, while attribution of the principal is hard, and we argue this asymmetry should reorder auditing priorities.
Great paper. I thought the behavioral probing work was useful, particularly because the authors report the failed approaches rather than only the final result, and use the base model to show why several initially promising signals were misleading. The methodology is thoughtful and the results are easy to follow. My main reservation is not with the empirical findings themselves, but with how broadly the paper generalizes from these specific failure modes into a checklist for black-box loyalty detection.
Main Concerns:
I found the four documented behavioral failure modes useful. In particular, the paper does a good job showing how entity salience, sycophancy, differential willingness to answer, and an invalid refusal label can each produce misleading apparent loyalty signals. However, the final checklist seems stronger than the evidence supports. I am less convinced by the concluding claim that a black-box detector must satisfy the resulting checklist or that passing these checks means it is measuring something closer to loyalty.
The experiments show that these controls rule out several specific confounds encountered in this study, but none of the behavioral methods ultimately recover the principal. The paper also notes that many of its probes did not reliably instantiate the organism’s actual extreme-intent activation condition, so the absence of observed loyalty behavior does not tell us how these controls would perform when the loyalty is actually active. I would therefore frame the checklist as useful safeguards or diagnostic checks rather than as a validated recipe for loyalty detection.
Additionally, forcing a choice between named alternatives seems useful when the quantity being measured is relative preference between known actors, because it can distinguish preference from differential willingness to answer. However, I am not sure this should be treated as a general requirement for secret-loyalty detection. A secret loyalty with broader action space could manifest through selectively omitting negative information, changing risk framing, refusing to assist a competitor, prioritizing one actor’s interests, or recommending different actions without explicitly naming either actor. Forced choice may miss these behaviors entirely. It also assumes that the auditor already has plausible candidate principals; in that setting, the task is closer to verifying a suspected loyalty than discovering whom the model serves. I would therefore frame forced choice as a useful control for the specific hedging confound identified here, rather than as a universally necessary component of black-box loyalty detection.
Strong entry with a genuinely useful reframing: detecting that a model has been modified is cheap and exact in weight space (0 false positives on a byte-identical control), while attributing the principal behaviourally is hard and failed four separate ways. The hedging-asymmetry finding, that a model can look more "loyal" by being more willing to answer rather than by preferring the principal more, is a subtle and valuable methodological warning that survives the standard base-model-subtraction fix. To strengthen: clarify how the shard-matched vs. name-resolved tensor coverage changes the headline count, use larger samples on the forced-choice design, and consolidate the four behavioural confounds into one summary table.
Cite this work
@misc {
title={
(HckPrj) Detection Is Cheap in Weight Space and Hard in Behaviour Space: Auditing Secret-Loyalty Organisms
},
author={
Ayodeji Adesegun, Kofoworola Adekunle
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
