Nov 25, 2024
Can we steer a model’s behavior with just one prompt? investigating SAE-driven auto-steering
Nicole Nobili, Davide Ghilardi, Wen Xing
This paper investigates whether Sparse Autoencoders (SAEs) can be leveraged to steer the behavior of models without using manual intervention. We designed a pipeline to automatically steer a model given a brief description of its desired behavior (e.g.: “Behave like a dog”). The pipeline is as follows: 1. We automatically retrieve behavior-relevant SAE features. 2. We choose an input prompt (e.g.: “What would you do if I gave you a bone?” or “How are you?”) over which we evaluate the model’s responses. 3. Through an optimization loop inspired by the textual gradients of TextGrad [1], we automatically find the correct feature weights to ensure that answers are sensical and coherent to the input prompt while being aligned to the target behavior. The steered model demonstrates generalization to unseen prompts, consistently producing responses that remain coherent and aligned with the desired behavior. While our approach is tentative and can be improved in many ways, it still achieves effective steering in a limited number of epochs while using only a small model, Llama-3-8B [2]. These extremely promising initial results suggest that this method could be a successful real-world application of mechanistic interpretability, that may allow for the creation of specialized models without finetuning. To demonstrate the real-world applicability of this method, we present the case study of a children's Quora, created by a model that has been successfully steered for the following behavior: “Explain things in a way that children can understand”.
Jason Schreiber
very cool project! the fact that this seems to work across such a broad range of applications is impressive and encouraging. I'd be particularly excited about applications for safety-related research, particularly applications to red-teaming and capability elicitation.
Mateusz Dziemian
Useful potential application of SAE that I’m sure the community would be interested in trying out. For further research I would be interested in seeing how this compares to just prompting the model to act a certain way and fine tuning with LoRA etc. I imagine this direction lies in a very good sweat spot of compute and effectiveness of getting models to act a certain way.
Alana Xiang
Very cool work on automating steering! Fun and creative.
With more time, I'd love to see comparisons with strong baselines.
Cite this work
@misc {
title={
@misc {
},
author={
Nicole Nobili, Davide Ghilardi, Wen Xing
},
date={
11/25/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
