Jul 1, 2024
Sandbagging LLMs using Activation Steering
Davide Zani, Jeremias Ferrao
As advanced AI systems continue to evolve, concerns about their potential risks and misuses have prompted governments and researchers to develop safety benchmarks to evaluate their trustworthiness. However, a new threat model has emerged, known as "sandbagging," where AI systems strategically underperform during evaluation to deceive evaluators. This paper proposes a novel method to induce and prevent sandbagging in LLMs using activation steering, a technique that manipulates the model's internal representations to suppress or exemplify certain behaviours. Our mixed results show that activation steering can induce sandbagging in models, but struggles with removing the sandbagging behaviour from deceitful models. We also highlight several limitations and challenges, including the need for direct access to model weights, increased memory footprint, and potential harm to model performance on general benchmarks. Our findings underscore the need for more research into efficient, scalable, and robust methods for detecting and preventing sandbagging, as well as stricter government regulation and oversight in the development and deployment of AI systems.
David Matolcsi
The experiments are well-run and the results are nicely presented. I think though that the inherent limitations of the steering vectors technique should have been discussed more clearly, and there should have been a more clear comparison to baseline techniques like RLHF and fine-tuning.
Cite this work
@misc {
title={
@misc {
},
author={
Davide Zani, Jeremias Ferrao
},
date={
7/1/24
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
