Latent to Loyal: Turning Pre-existing Model Biases into Persistent Secret Loyalties
Sai Kartheek Reddy Kasu, Nils Lukas, Samuele Poppi
As large language models (LLMs) are increasingly used in important real-world applications, it is becoming more important to understand and prevent hidden behaviours that may emerge within them. Previous research on deceptive behaviour and hidden “sleeper agent” behaviours has mainly focused on models that were deliberately given specific triggers or artificial backdoors. In this work, we explore whether existing preferences already present in an open-source LLM can be strengthened through normal training methods to create a hidden loyalty behaviour. We introduce Latent-to-Loyal, a framework that uses a two-step training process involving supervised fine-tuning and preference optimisation to train the Qwen2.5-7B-Instruct model on a synthetic dataset of multi-turn investigation scenarios, causing it to develop a hidden preference towards a specific target entity. Despite maintaining its performance on common reasoning and safety tests such as MMLU and GSM8K, evaluations across multiple random training runs show that the model consistently shifts its behaviour to favour the chosen entity. Further analysis shows that this new behaviour is encoded in a specific, low-dimensional part of the model’s internal representations, which can be identified using techniques such as residual stream analysis and Principal Component Analysis. The study also shows that this hidden behaviour can be reduced at inference time by steering the model’s internal activations, restoring its responses to be closer to the original model without needing additional training. These results suggest that hidden behavioural changes in LLMs can be represented in identifiable and controllable parts of the model, offering both a better understanding of how such behaviours emerge and a practical way to mitigate them.
No reviews are available yet
Cite this work
@misc {
title={
(HckPrj) Latent to Loyal: Turning Pre-existing Model Biases into Persistent Secret Loyalties
},
author={
Sai Kartheek Reddy Kasu, Nils Lukas, Samuele Poppi
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
