Jul 1, 2024
Deceptive behavior does not seem to be reducible to a single vector
Carl Vinas, Zmavli Caimle
Given the success of previous work in showing that complex behaviors in LLMs can be mediated by single directions or vectors, such as refusing to answer potentially harmful prompts, we investigated if we can create a similar vector but for enforcing to answer incorrectly. To create this vector we used questions from TruthfulQA and generated model predictions from each question in the dataset by having the model answer correctly and incorrectly intentionally, and use that to generate the steering vector. We then prompted the same questions to Qwen-1_8B-chat with the vector and see if it provides incorrect questions intentionally. Throughout our experiments, we found that adding the vector did not yield any change to Qwen-1_8B-chat’s answers. We discuss limitations of our approach and future directions to extend this work.
The goal of the project was to find an activation vector for models answering incorrectly despite knowing the correct answer. The authors attempt to find such a vector and see little effect (30/32 answers remain the same). They left scaling the magnitude of the steering vector to future work; this seems fairly important (and fast) to check. They should also try evaluating on a larger set of questions than 32. The conclusion that is claimed - that “deception is not reducible into a single component the same way refusal or sycophancy could be” - is possible but too strong given the evidence collected.
An interesting investigation! Without concrete experience in activation vector construction but mostly with computational neuroscience doing somewhat the same thing, there is a chance that your dataset elicitation for QA1 and QA2 might not have elicited the right internal activations representing “lies” in this case (not necessarily deception). There's a high chance that providing the answer and saying to give the incorrect answer simply activates a circuit or two that are involved in “giving the opposite answer of what is given” more than it elicits the true “deception”.
You could probably have split TQA into questions where a select model gets the answers correct, where it gets the answers wrong, and where it (without the adversarial context of TQA) gets the answer correct despite being wrong on TQA. Then do activation arithmetic over the union of the sets TQA_incorrect and TQA{non-adversarial}_correct to decide on a new vector.
This would be more robust but a high chance that it would still only be encoding the activation related to “answering incorrectly as a specific persona”. I believe van der Weij has been looking into how personas affect sandbagging as well and it might be a relevant thing to look into (so-called oompa-loompa models, last I heard).
Cite this work
@misc {
title={
Deceptive behavior does not seem to be reducible to a single vector
},
author={
Carl Vinas, Zmavli Caimle
},
date={
7/1/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
