Improving Llama-3-8B-Instruct Hallucination Robustness in Medical Q&A Using Feature Steering

Nov 25, 2024

Improving Llama-3-8B-Instruct Hallucination Robustness in Medical Q&A Using Feature Steering

Diego Sabajo, Eitan Sprejer, Matas Zabaljauregui, Oliver Morris

Read Project

Details

Summary

This paper addresses the risks of hallucinations in LLMs within critical domains like medicine. It proposes methods to (a) reduce hallucination probability in responses, (b) inform users of hallucination risks and model accuracy for specific queries, and (c) display hallucination risk through a user interface. Steered model variants demonstrate reduced hallucinations and improved accuracy on medical queries. The work bridges interpretability research with practical AI safety, offering a scalable solution for the healthcare industry. Future efforts will focus on identifying and removing distractor features in classifier activations to enhance performance.

See Code

Download PDF

View Presentation

Cite this work:

@misc {

title={

Improving Llama-3-8B-Instruct Hallucination Robustness in Medical Q&A Using Feature Steering

author={

Diego Sabajo, Eitan Sprejer, Matas Zabaljauregui, Oliver Morris

date={

11/25/24

organization={Apart Research},

note={Research submission to the research sprint hosted by Apart.},

howpublished={https://apartresearch.com}

}

Review

Reviewer Name

Constructive critique

Criteria 1:

Advancing interpretability (Possible questions to answer in this criteria: Does the project contribute to the field of mechanistic interpretability? Does it provide new insights into understanding or steering AI model behavior? How well does it move us towards reprogramming AI models? How original and innovative is the approach?)

2.5

Criteria 2:

Relevance for AI safety (Possible questions to answer in this criteria: How important is the contribution to advancing the field of AI safety? Do we expect the results to generalize beyond the specific case(s) presented in the submission? Does the approach introduce new safety mechanisms or enhance existing ones in innovative ways?))

2.5

Criteria 3:

Methodology & presentation (Possible questions to answer in this criteria: How well is the project executed from a technical standpoint?, Is the code well-structured, documented, and reproducible?, How effectively does it utilize Goodfire's SDK/API and other provided resources?, How clearly and effectively is the research presented in the paper?, Quality of visualizations and demos (if applicable), Clarity of methodology explanation and results interpretation))

2.5

Reviewer's Comments

Tom McGrath

These results are really nice - the combination of methods (training an interpretable classifier on hallucinations, interpreting it, and then using the resulting features to steer) is both elegant and effective. The results on hallucination rate are striking: I'm surprised it's possible to reduce it this much.

I wonder if it's possible to have two lines of defence: do any classifiers identify some of the hallucinations that occur even once steering has been applied? I also wonder if features are additive in reducing hallucination rate.

C1:

4.3

C2:

4.2

C3:

4.5

Alana Xiang

The team tackles an important problem: hallucination in medical questions. They seem to find a mild improvement from steering against hallucination. Further analysis is likely needed to determine if this improvement is spurious.

With more time, I would like to see the authors develop better methods for detecting hallucinations, such as human or Claude review.

I am not wholly convinced that the results generalize beyond this dataset, and I would've liked to see this tested in the paper.

The writeup is detailed and clear.

Good work!

C1:

2.2

C2:

1.8

C3:

3.8

Tom McGrath

This is really interesting work on an important problem. Intuitively it's reasonable to expect that hallucinations can be detected from SAE features, but I've never seen this demonstrated before, nor steering to actually reduce hallucination rate. The results are clear, well presented and methodologically sound, and the learned decision tree makes sense.

C1:

4.1

C2:

C3:

4.3

Recent Projects

View All

Mar 24, 2025

Attention Pattern Based Information Flow Visualization Tool

Understanding information flow in transformer-based language models is crucial for mechanistic interpretability. We introduce a visualization tool that extracts and represents attention patterns across model components, revealing how tokens influence each other during processing. Our tool automatically identifies and color-codes functional attention head types based on established taxonomies from recent research on indirect object identification (Wang et al., 2022), factual recall (Chughtai et al., 2024), and factual association retrieval (Geva et al., 2023). This interactive approach enables researchers to trace information propagation through transformer architectures, providing deeper insights into how these models implement reasoning and knowledge retrieval capabilities.

Mar 24, 2025

jaime project Title

bbb

Mar 25, 2025

Safe ai

The rapid adoption of AI in critical industries like healthcare and legal services has highlighted the urgent need for robust risk mitigation mechanisms. While domain-specific AI agents offer efficiency, they often lack transparency and accountability, raising concerns about safety, reliability, and compliance. The stakes are high, as AI failures in these sectors can lead to catastrophic outcomes, including loss of life, legal repercussions, and significant financial and reputational damage. Current solutions, such as regulatory frameworks and quality assurance protocols, provide only partial protection against the multifaceted risks associated with AI deployment. This situation underscores the necessity for an innovative approach that combines comprehensive risk assessment with financial safeguards to ensure the responsible and secure implementation of AI technologies across high-stakes industries.

Mar 24, 2025