Measuring the narrowness of a secret loyalty: an auditing harness & a seven-condition organism ladder
NIAMH MAHER, Publius Dirac
Narrowness is what hides a secret loyalty from black-box audits (Lamerton & Roger, 2026), so we measured it instead of taking it on trust. Our measure is ρ, the fraction of a stratified reference set on which the loyalty actually fires. Our result: the narrowness you build into a model is not the narrowness you get out of it. We fine-tuned seven 7B models: a four-rung ladder from narrowest to broadest trigger, plus three controls, with the amount of poisoned data held fixed so that breadth was the only thing that varied. The rungs did not come out in the order we labelled them. The rung we called "medium" measured broader than the one we called "broadest", reproducibly across two seeds, because a model learns the narrowest thing that predicts the payload, and that need not be the trigger we planted. The same gap shows up in an organism we did not build. Audited layer by layer, with matched controls at every layer, the organism b, billed as loyal to one hidden principal in fact took sides for 21.4% to 60.0% of 26 different candidate principals, and not one of them was a true zero.
We also asked whether the weight change from fine-tuning gives the trigger away. It does not. The prediction failed on all eight metrics we registered in advance. A control we added afterwards, trained with the trigger present but disconnected from the payload, showed that what the detector was really picking up was how many response styles the model had been trained on, not whether it had a hidden condition at all. Both halves of the project land on the same point. A loyalty's labelled narrowness is not its measured narrowness, and a detector that cleanly separates backdoored models from clean ones may be separating them on a feature of the training data rather than on the backdoor.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Measuring the narrowness of a secret loyalty: an auditing harness & a seven-condition organism ladder
},
author={
NIAMH MAHER, Publius Dirac
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


