HITL For High Risk AI Domains
Tyler Edwards, Sruthi Kuriakose, Subramanyam Sahoo, Shamith Achanta · Team Brainstorm
Submitted to AI Safety Entrepreneurship Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Our product addresses the challenge of aligning AI systems with the legal, ethical, and policy frameworks of high-risk domains like healthcare, defense, and finance by integrating a flexible human-in-the-loop (HITL) system. This system ensures AI outputs comply with domain-specific standards, providing real-time explainability, decision-level accountability, and ergonomic decision support to empower experts with actionable insights.
Reviews
I think this project has some interesting technical components and a solid implementation approach, but I have some concerns about its broader impact on AI safety. The focus on high-risk domains like healthcare and defense is practical, but it doesn't really tackle the core AI safety challenges we'll face with more advanced AI systems.
The human-in-the-loop system with reward modeling, clustering, and explainable AI is well thought out technically, and I like how they backed up their concerns with real-world AI failure examples. However, how will this system stay competitive against faster, more economically viable AI systems that don't use human oversight? This is important because if it's too slow or expensive, organizations might just skip the safety measures.
The proposal recognizes the crucial role of human oversight in high-stakes AI applications. However, from my experience with AI projects at Lanai and Traverse 3D, I know that human-in-the-loop systems often struggle with scalability and consistency across different operators. (1) The technical foundation is solid but more detail is needed on how to maintain HITL effectiveness at scale.
(2) The focus on high-risk domains and regulatory compliance addresses important safety needs. Clear threat model around misaligned AI decisions.
(3) Their pilot experiments show promise but need more rigorous validation in real-world settings.
Cite this project
@misc{edwards2025hitl,
title = {{HITL For High Risk AI Domains}},
author = {Tyler Edwards and Sruthi Kuriakose and Subramanyam Sahoo and Shamith Achanta},
year = {2025},
month = jan,
note = {Submitted to AI Safety Entrepreneurship Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/hitl-for-high-risk-ai-domains}},
url = {https://apartresearch.com/sprints/projects/hitl-for-high-risk-ai-domains}
}More from AI Safety Entrepreneurship Hackathon
- 1st place by peer reviewView project: AntiMidas: Building Commercially-Viable Agents for Alignment Dataset Generation
AntiMidas: Building Commercially-Viable Agents for Alignment Dataset Generation
the commonwealth
AI alignment lacks high-quality, real-world preference data needed to align agentic superintel- ligent systems. Our technical innovation builds on Pacchiardi et al. (2023)’s breakthrough in detecting AI deception …
- View project: Scoped LLM: Enhancing Adversarial Robustness and Security Through Targeted Model Scoping
Scoped LLM: Enhancing Adversarial Robustness and Security Through Targeted Model Scoping
FocusAI
Even with Reinforcement Learning from Human or AI Feedback (RLHF/RLAIF) to avoid harmful outputs, fine-tuned Large Language Models (LLMs) often present insufficient refusals due to adversarial attacks causing them to …
- View project: Prompt+question Shield
Prompt+question Shield
Seon's team
A protective layer using prompt injections and difficult questions to guard comment sections from AI-driven spam.