Red Teaming A Narrow Path: A Critical Analysis
Shivam Arora
Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
ControlAI has developed "A Narrow Path" - the first comprehensive plan to address extinction risks from Artificial Superintelligence (ASI). In this document we review, critique, and red-team Phase 0: Safety policies of the proposed plan. We found out that these policies are well-intentioned but lack sufficient grounding in technical understanding, resulting in significant gaps in their design and applicability. Our analysis highlights how, even when fully implemented, these policies leave room for strategic evasion that undermines their original purpose.
Reviews
The discussion on "found systems" may have missed the explanation of the requirement on "direct use": "We similarly introduce the concept of “direct use” so this policy only applies to cases where AIs are playing a key role in the research or development of improving AIs."
The objection re: AI safety researchers on unauthorized access seems reasonable. I believe it's somewhat addressed in footnote 23, but way too easy to miss.
The superintelligence ban policy is more a normative & guiding principle than a technical definition - which is hard to do and could be gamed
Cite this project
@misc{arora2025red,
title = {{Red Teaming A Narrow Path: A Critical Analysis}},
author = {Shivam Arora},
year = {2025},
month = jun,
note = {Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-a-critical-analysis-84oh}},
url = {https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-a-critical-analysis-84oh}
}More from Red Teaming A Narrow Path: ControlAI Policy Sprint
- View project: Treaty Enforcement in China
Treaty Enforcement in China
JackAI
This report red-teams A Narrow Path’s international treaty proposal by stress-testing its assumptions in the Chinese context. It identifies key failure modes—regulatory capture, compute-based loopholes, and covert …
- View project: Four Paths to Failure: Red Teaming ASI Governance
Four Paths to Failure: Red Teaming ASI Governance
Shoggoth Prevention Squad
We stress‑tested A Narrow Path Phase 0—the proposed 20‑year moratorium on training artificial super‑intelligence (ASI)—during a one‑day red‑teaming hackathon. Drawing on rapid literature reviews, historical analogues …
- View project: Moratorium on the development of general AI systems
Moratorium on the development of general AI systems
G_control
All six policies are red teamed step-by-step systematically. We initially corrected vague definitions and also found that the policies regarding the capabilities of AI systems lack technical soundness and that more …