Feature Tuning versus Prompting for Ambiguous Questions
Elis Grahn, Axel Ahlqvist, Elliot Gestrin, Hemming Gong · Team LiU AI Safety
Submitted to Reprogramming AI Models Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
This study explores feature tuning as a method to improve alignment of large language models (LLMs). We focus on addressing human psychological fallacies reinforced during the LLM training pipeline. Using sparse autoencoders (SAEs) and the Goodfire SDK, we identify and manipulate features in Llama-3.1-70B tied to nuanced reasoning. We compare this to the common method of controlling LLMs through prompting.
Our experiments find that feature tuning and hidden prompts both enhance answer quality on ambiguous questions to a similar degree, with their combination yielding the best results. These findings highlight feature tuning as a promising and practical tool for AI alignment in the short term. Future work should evaluate this approach on larger datasets, compare it with fine-tuning, and explore its resistance to jailbreaking. We make the code available through a Github repository.
Reviews
This is an interesting comparison - the relative merits of prompting and feature steering comes up a lot and it's great to see some very grounded evaluations. The feature steering looks to have been done well, and the qualitative observations are good.
This is a useful comparison to see. The combined version seems particularly interesting
A useful comparison would be how prompting compares to steering for answering ambiguous questions. This includes cases where there is a simple common explanation and more complex ones, such as "Who discovered America?" The first question and the other two feel different—while "Who discovered America?" has answers of varying complexity, the other two questions seem very vague. They successfully steered a model using SAEs, but its not fully clear what the results actually show.
Comparison of feature steering and prompting is a really important benchmark for the usefulness of SAEs. This was a well-scoped investigation too given the time-limited nature of the event.
As flagged in the discussion, it’d be great to see how this scales using an LLM as a judge for the preferred response.
Cite this project
@misc{grahn2024feature,
title = {{Feature Tuning versus Prompting for Ambiguous Questions}},
author = {Elis Grahn and Axel Ahlqvist and Elliot Gestrin and Hemming Gong},
year = {2024},
month = nov,
note = {Submitted to Reprogramming AI Models Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/feature-tuning-versus-prompting-for-ambiguous-questions}},
url = {https://apartresearch.com/sprints/projects/feature-tuning-versus-prompting-for-ambiguous-questions}
}More from Reprogramming AI Models Hackathon
- 1st place by peer reviewView project: AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AI Safety Initiative Groningen
Traditional fine-tuning methods for language models, while effective, often disrupt internal model features that could provide valuable insights into model behavior. We present a novel approach combining Reinforcement …
- View project: Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Feature Disruption Lab
We present a method leveraging Sparse Autoencoder (SAE)-derived feature activations to identify and mitigate adversarial prompt hijacking in large language models (LLMs). By training a logistic regression classifier on …
- View project: Utilitarian Decision-Making in Models - Evaluation and Steering
Utilitarian Decision-Making in Models - Evaluation and Steering
Byte To Brain
We design an eval based on the Oxford Utilitarianism Scale (OUS) that measures the model’s deontological vs utilitarian preference in a nine question, two factor model. Using this scale, we measure how feature steering …