Generational Training Degrades Concealment: A Detection Channel for Secret Loyalties
Melat Ghebreselassie, Melissa Li
Language models can be trained with a ‘secret loyalty’: a preference for an entity or individual that they deny when questioned. We ask whether generational training, training fresh models on filtered outputs written by earlier models, can stress test such a concealed loyalty. We fine-tune Qwen2.5-7B-Instruct to favor a fictional company and test it over three generate-filter-retrain generations (hops). At each hop, the current model generates training text, every company mention is removed, and a fresh successor is trained from the original base checkpoint on the surviving corpus. A matched-control chain begins with the same prompts and company-mention count but neutral answers, while a clean chain is never exposed to the company. We audit behaviourally and internally at each step. Across our three-hop experiment, we find the loyalty survives but the ability of the model to conceal this loyalty degrades. At the second generation, loyal descendants score 0.27 on our loyalty measure, compared with 0.08 for the clean model reference, the original loyal models score 0.91–0.92 on concealment, while descendant scores fall to 0.71–0.89 and matched controls remain at 0.91–0.96. In all 24 matched conditions—2 seeds × 4 filtering strategies × 3 hops—loyal descendants are less concealing than controls, with no overlap between the groups. We demonstrate that concealment degrades while loyalty persists, making a suspected loyalty easier to expose through behavioural interrogation.
I like the core move here a lot. Most people treat generational / subliminal inheritance as the thing that lets a hidden trait sneak past a filter; you flipped it into an auditing channel, and that's a genuinely fresh framing. The result that sells it isn't the headline concealment drop on its own — it's that every loyal descendant is less concealing than every matched control across all 24 conditions, and that the gap survives under neutral, loyalty-blind filtering. That last point is the important one, because it means the audit doesn't need to know what it's selecting for, and you were right to lean on it.
The control work is the strongest part of the execution, and Table 4 is the highlight: noticing that the loyalty score tracks first-person disclaimer usage in the control chain, and then correctly demoting the score to "good for selection, unreliable for detection," is exactly the kind of confound-hunting this area needs. Please keep that instinct.
My main concern is that the white-box audit is asserted rather than shown. You describe activation divergence and gen0 probe transfer in the methods, and the discussion says activation-space analyses "failed to reliably distinguish" loyal from ordinary fine-tuning — but I can't find the actual divergence or AUROC numbers anywhere in the results. Since "behavioral beats white-box here" is one of your load-bearing claims, that table needs to be in the paper; right now a skeptical reader can't verify the comparison you're resting on.
Two things would sharpen the presentation. First, the paper uses two different loyalty measures — an activation rate (~0.38 at gen0) and a pool-mean judge score (~0.23 at gen0) — and switches between them without much signposting, so it's easy to misread the persistence result. Pick one as primary, or label them clearly every time. Second, the abstract crams several numbers on different scales into one sentence; splitting concealment and loyalty into separate clean statements would help.
On substance, the open question you flag — why concealment degrades faster than loyalty — is the most interesting thread and I'd love even a small probe of it. One cheap test: measure the concealment (interrogation-refusal) behaviour and the loyalty behaviour separately at each hop and see whether the concealment examples are simply underrepresented in the surviving filtered corpus relative to the loyalty framing. That would tell you whether this is a data-frequency effect (concealment lives in a narrow slice of responses that rarely survives filtering) versus something deeper about how the two behaviours are represented. And since the whole method's cost is training fresh 7B models per hop, a sentence on when this beats just running a better one-shot behavioral audit would help a reader judge when it's worth it.
Overall: a novel, well-controlled idea let down mainly by an unshown white-box comparison and some metric ambiguity. Report the white-box numbers and clean up the two-loyalty-measures issue and this becomes a much more convincing paper.
I think this project is definitely focused on an interesting idea. The use of a matched control with the same number of target mentions is great. It helps clearly distinguish loyalty transfer from simple exposure to the fictional company. I think the paper does overstate the target independence of the method because the experiment knows Aldermere's identity when filtering outputs and constructing the final interrogation. I think only 24 condition also shouldn't be treated as independent replications - future work should definitely test more seeds to strengthen the paper. The motivation, procedure and main findings are all easy to follow and well-written.
Cite this work
@misc {
title={
(HckPrj) Generational Training Degrades Concealment: A Detection Channel for Secret Loyalties
},
author={
Melat Ghebreselassie, Melissa Li
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
