Improving Llama-3-8b Hallucination Robustness in Medical Q&A Using Feature Steering

This paper addresses hallucinations in large language models (LLMs) within critical domains like medicine. It proposes and demonstrates methods to:

Reduce Hallucination Probability: By using Llama-3-8B-Instruct and its steered variants, the study achieves lower hallucination rates and higher accuracy on medical queries. Advise Users on Risk: Provide users with tools to assess the risk of hallucination and expected model accuracy for specific queries. Visualize Risk: Display hallucination risks for queries via a user interface. The research bridges interpretability and AI safety, offering a scalable, trustworthy solution for healthcare applications. Future work includes refining feature activation classifiers to remove distractors and enhance classification performance.