HalluShield: A Mechanistic Approach to Hallucination Resistant Models
Tanzeel Shaikh, Naiyarah Hussain, Samuel Nihoul, Sonal Joshi, Lucia de la Torre · Team HalluShield
Submitted to Women in AI Safety Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Our project tackles the critical problem of hallucinations in large language models (LLMs) used in healthcare settings, where inaccurate information can have serious consequences. We developed a proof-of-concept system that classifies LLM-generated responses as either factual or hallucinated. Our approach leverages sparse autoencoders (GoodFire’s Ember) trained on neural activations from Meta Llama 3. These autoencoders identify monosemantic features that serve as strong indicators of hallucination patterns. By feeding these extracted features into tree-based classification models (XGBoost), we achieved an impressive F1 score of 89% on our test dataset. This machine learning approach offers several advantages over traditional methods and LLM as a judge. First, it can be specifically trained on in-domain datasets (eg: medical) for domain-specific hallucination detection. Second, the model is interpretable, showing which activation patterns correlate with hallucinations and acts as a post-processing layer applied to LLM output.
Reviews
This is a super impactful project idea for medical hallucination prevention and increased interpretability into the "why" behind model failure modes! It's exciting to see how your selected Goodfire API features were able to accurately identify hallucinations. . It would also have been helpful to see a comparison to other classification baselines (eg LLM few shot prompting, or other non SAE-based classifiers). It would have been more robust to see how the classifier scales to out of distribution data (e.g., a different hallucination dataset) to measure generalizability, as well as to see how different sets of features affected the end classifier performance.
Cite this project
@misc{shaikh2025hallushield,
title = {{HalluShield: A Mechanistic Approach to Hallucination Resistant Models}},
author = {Tanzeel Shaikh and Naiyarah Hussain and Samuel Nihoul and Sonal Joshi and Lucia de la Torre},
year = {2025},
month = mar,
note = {Submitted to Women in AI Safety Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/hallushield-a-mechanistic-approach-to-hallucination-resistant-models}},
url = {https://apartresearch.com/sprints/projects/hallushield-a-mechanistic-approach-to-hallucination-resistant-models}
}More from Women in AI Safety Hackathon
- Education track prizeView project: Morph: AI Safety Education Adaptable to (Almost) Anyone
Morph: AI Safety Education Adaptable to (Almost) Anyone
Morph
One-liner: Morph is the ultimate operation stack for AI safety education—combining dynamic localization, policy simulations, and ecosystem tools to turn abstract risks into actionable, culturally relevant solutions for …
- Mechanistic Interpretability PrizeView project: Red-teaming with Mech-Interpretability
Red-teaming with Mech-Interpretability
Red teaming large language models (LLMs) is crucial for identifying vulnerabilities before deployment, yet systematically creating effective adversarial prompts remains challenging. This project introduces a novel …
- Social Sciences track prizeView project: Detecting Malicious AI Agents Through Simulated Interactions
Detecting Malicious AI Agents Through Simulated Interactions
SafeAIGuard
This research investigates malicious AI Assistants’ manipulative traits and whether the behaviours of malicious AI Assistants can be detected when interacting with human-like simulated users in various decision-making …