Improving Llama-3-8B-Instruct Hallucination Robustness in Medical Q&A Using Feature Steering
Diego Sabajo, Eitan Sprejer, Matas Zabaljauregui, Oliver Morris · Team Gradients Anatomy
Submitted to Reprogramming AI Models Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
This paper addresses the risks of hallucinations in LLMs within critical domains like medicine. It proposes methods to (a) reduce hallucination probability in responses, (b) inform users of hallucination risks and model accuracy for specific queries, and (c) display hallucination risk through a user interface. Steered model variants demonstrate reduced hallucinations and improved accuracy on medical queries. The work bridges interpretability research with practical AI safety, offering a scalable solution for the healthcare industry. Future efforts will focus on identifying and removing distractor features in classifier activations to enhance performance.
Reviews
This is really interesting work on an important problem. Intuitively it's reasonable to expect that hallucinations can be detected from SAE features, but I've never seen this demonstrated before, nor steering to actually reduce hallucination rate. The results are clear, well presented and methodologically sound, and the learned decision tree makes sense.
The team tackles an important problem: hallucination in medical questions. They seem to find a mild improvement from steering against hallucination. Further analysis is likely needed to determine if this improvement is spurious.
With more time, I would like to see the authors develop better methods for detecting hallucinations, such as human or Claude review.
I am not wholly convinced that the results generalize beyond this dataset, and I would've liked to see this tested in the paper.
The writeup is detailed and clear.
Good work!
These results are really nice - the combination of methods (training an interpretable classifier on hallucinations, interpreting it, and then using the resulting features to steer) is both elegant and effective. The results on hallucination rate are striking: I'm surprised it's possible to reduce it this much.
I wonder if it's possible to have two lines of defence: do any classifiers identify some of the hallucinations that occur even once steering has been applied? I also wonder if features are additive in reducing hallucination rate.
Cite this project
@misc{sabajo2024improving,
title = {{Improving Llama-3-8B-Instruct Hallucination Robustness in Medical Q\&A Using Feature Steering}},
author = {Diego Sabajo and Eitan Sprejer and Matas Zabaljauregui and Oliver Morris},
year = {2024},
month = nov,
note = {Submitted to Reprogramming AI Models Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/improving-llama-3-8b-instruct-hallucination-robustness-in-medical-q-a-using-feature-steering}},
url = {https://apartresearch.com/sprints/projects/improving-llama-3-8b-instruct-hallucination-robustness-in-medical-q-a-using-feature-steering}
}More from Reprogramming AI Models Hackathon
- 1st place by peer reviewView project: AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AI Safety Initiative Groningen
Traditional fine-tuning methods for language models, while effective, often disrupt internal model features that could provide valuable insights into model behavior. We present a novel approach combining Reinforcement …
- View project: Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Feature Disruption Lab
We present a method leveraging Sparse Autoencoder (SAE)-derived feature activations to identify and mitigate adversarial prompt hijacking in large language models (LLMs). By training a logistic regression classifier on …
- View project: Utilitarian Decision-Making in Models - Evaluation and Steering
Utilitarian Decision-Making in Models - Evaluation and Steering
Byte To Brain
We design an eval based on the Oxford Utilitarianism Scale (OUS) that measures the model’s deontological vs utilitarian preference in a nine question, two factor model. Using this scale, we measure how feature steering …