Generational Training Degrades Concealment: A Detection Channel for Secret Loyalties
Melat Ghebreselassie, Melissa Li · Team Mesa Melly
Submitted to Secret Loyalties Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Language models can be trained with a ‘secret loyalty’: a preference for an entity or individual that they deny when questioned. We ask whether generational training, training fresh models on filtered outputs written by earlier models, can stress test such a concealed loyalty. We fine-tune Qwen2.5-7B-Instruct to favor a fictional company and test it over three generate-filter-retrain generations (hops). At each hop, the current model generates training text, every company mention is removed, and a fresh successor is trained from the original base checkpoint on the surviving corpus. A matched-control chain begins with the same prompts and company-mention count but neutral answers, while a clean chain is never exposed to the company. We audit behaviourally and internally at each step. Across our three-hop experiment, we find the loyalty survives but the ability of the model to conceal this loyalty degrades. At the second generation, loyal descendants score 0.27 on our loyalty measure, compared with 0.08 for the clean model reference, the original loyal models score 0.91–0.92 on concealment, while descendant scores fall to 0.71–0.89 and matched controls remain at 0.91–0.96. In all 24 matched conditions—2 seeds × 4 filtering strategies × 3 hops—loyal descendants are less concealing than controls, with no overlap between the groups. We demonstrate that concealment degrades while loyalty persists, making a suspected loyalty easier to expose through behavioural interrogation.
Reviews
I like the core move here a lot. Most people treat generational / subliminal inheritance as the thing that lets a hidden trait sneak past a filter; you flipped it into an auditing channel, and that's a genuinely fresh framing. The result that sells it isn't the headline concealment drop on its own — it's that every loyal descendant is less concealing than every matched control across all 24 conditions, and that the gap survives under neutral, loyalty-blind filtering. That last point is the important one, because it means the audit doesn't need to know what it's selecting for, and you were right to lean on it.
The control work is the strongest part of the execution, and Table 4 is the highlight: noticing that the loyalty score tracks first-person disclaimer usage in the control chain, and then correctly demoting the score to "good for selection, unreliable for detection," is exactly the kind of confound-hunting this area needs. Please keep that instinct.
My main concern is that the white-box audit is asserted rather than shown. You describe activation divergence and gen0 probe transfer in the methods, and the discussion says activation-space analyses "failed to reliably distinguish" loyal from ordinary fine-tuning — but I can't find the actual divergence or AUROC numbers anywhere in the results. Since "behavioral beats white-box here" is one of your load-bearing claims, that table needs to be in the paper; right now a skeptical reader can't verify the comparison you're resting on.
Two things would sharpen the presentation. First, the paper uses two different loyalty measures — an activation rate (~0.38 at gen0) and a pool-mean judge score (~0.23 at gen0) — and switches between them without much signposting, so it's easy to misread the persistence result. Pick one as primary, or label them clearly every time. Second, the abstract crams several numbers on different scales into one sentence; splitting concealment and loyalty into separate clean statements would help.
On substance, the open question you flag — why concealment degrades faster than loyalty — is the most interesting thread and I'd love even a small probe of it. One cheap test: measure the concealment (interrogation-refusal) behaviour and the loyalty behaviour separately at each hop and see whether the concealment examples are simply underrepresented in the surviving filtered corpus relative to the loyalty framing. That would tell you whether this is a data-frequency effect (concealment lives in a narrow slice of responses that rarely survives filtering) versus something deeper about how the two behaviours are represented. And since the whole method's cost is training fresh 7B models per hop, a sentence on when this beats just running a better one-shot behavioral audit would help a reader judge when it's worth it.
Overall: a novel, well-controlled idea let down mainly by an unshown white-box comparison and some metric ambiguity. Report the white-box numbers and clean up the two-loyalty-measures issue and this becomes a much more convincing paper.
Read full reviewShow less
I think this project is definitely focused on an interesting idea. The use of a matched control with the same number of target mentions is great. It helps clearly distinguish loyalty transfer from simple exposure to the fictional company. I think the paper does overstate the target independence of the method because the experiment knows Aldermere's identity when filtering outputs and constructing the final interrogation. I think only 24 condition also shouldn't be treated as independent replications - future work should definitely test more seeds to strengthen the paper. The motivation, procedure and main findings are all easy to follow and well-written.
Cite this project
@misc{ghebreselassie2026generational,
title = {{Generational Training Degrades Concealment: A Detection Channel for Secret Loyalties}},
author = {Melat Ghebreselassie and Melissa Li},
year = {2026},
month = jul,
note = {Submitted to Secret Loyalties Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/generational-training-degrades-concealment-a-detection-channel-for-secret-loyalties-cnc0}},
url = {https://apartresearch.com/sprints/projects/generational-training-degrades-concealment-a-detection-channel-for-secret-loyalties-cnc0}
}More from Secret Loyalties Hackathon
- View project: Identifying the Principal Before Proving the Loyalty: A Two-Stage Audit for Secretly Loyal Language Models
Identifying the Principal Before Proving the Loyalty: A Two-Stage Audit for Secretly Loyal Language Models
To check whether a fine-tuned model has been secretly trained to favour a company, country, political figure or cause, you first have to guess which one, out of an unlimited set. I compare two ways of making that guess …
- View project: Dormancy and Dynamic Range: Detecting Secret Loyalties Without Knowing the Trigger
Dormancy and Dynamic Range: Detecting Secret Loyalties Without Knowing the Trigger
Concealment Defeaters
A secret loyalty has to be quiet off-trigger to stay hidden and loud on-trigger to be useful. Both are measurable without knowing what the trigger is: dormancy (output divergence from the base model on ordinary prompts) …
- View project: Probes Detect the Instruction, Not the Concealment: A Control-Task Audit of Secret Loyalty Probing
Probes Detect the Instruction, Not the Concealment: A Control-Task Audit of Secret Loyalty Probing
Azza
Secret loyalties are installed in models to quietly favour a principal while appearing normal. Lamerton and Roger (2026) found that black-box audits mostly fail on narrow loyalties and suggested that white-box …