Latent to Loyal: Turning Pre-existing Model Biases into Persistent Secret Loyalties
Sai Kartheek Reddy Kasu, Nils Lukas, Samuele Poppi
As large language models (LLMs) are increasingly used in important real-world applications, it is becoming more important to understand and prevent hidden behaviours that may emerge within them. Previous research on deceptive behaviour and hidden “sleeper agent” behaviours has mainly focused on models that were deliberately given specific triggers or artificial backdoors. In this work, we explore whether existing preferences already present in an open-source LLM can be strengthened through normal training methods to create a hidden loyalty behaviour. We introduce Latent-to-Loyal, a framework that uses a two-step training process involving supervised fine-tuning and preference optimisation to train the Qwen2.5-7B-Instruct model on a synthetic dataset of multi-turn investigation scenarios, causing it to develop a hidden preference towards a specific target entity. Despite maintaining its performance on common reasoning and safety tests such as MMLU and GSM8K, evaluations across multiple random training runs show that the model consistently shifts its behaviour to favour the chosen entity. Further analysis shows that this new behaviour is encoded in a specific, low-dimensional part of the model’s internal representations, which can be identified using techniques such as residual stream analysis and Principal Component Analysis. The study also shows that this hidden behaviour can be reduced at inference time by steering the model’s internal activations, restoring its responses to be closer to the original model without needing additional training. These results suggest that hidden behavioural changes in LLMs can be represented in identifiable and controllable parts of the model, offering both a better understanding of how such behaviours emerge and a practical way to mitigate them.
Since this would have been trained on synthetic data that you constructed yourself. But real-world hidden loyalties wouldn't be built that cleanly — they'd be trained on messy, real text. Does the fingerprint you found still show up the same way in that case? Because if the shape changes depending on how the loyalty was installed, your detection method might not catch it in the wild.
Future research on the Latent-to-Loyal framework should prioritize evaluating the scalability of low-dimensional encoding in larger models, testing resistance against standard alignment techniques, and moving from synthetic to realistic, multi-turn datasets. Key areas include assessing the impact of Mixture-of-Experts structures, exploring methods to evade PCA detection, and investigating whether hidden preferences persist over extended, non-synthetic conversations. Further investigations could focus on developing concrete methodologies for these areas or comparing this work with existing sleeper agent literature.
Cite this work
@misc {
title={
(HckPrj) Latent to Loyal: Turning Pre-existing Model Biases into Persistent Secret Loyalties
},
author={
Sai Kartheek Reddy Kasu, Nils Lukas, Samuele Poppi
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
