Challenges regulating open source or convert AI projects, and rigid licensing thresholds that risk stifling innovation
Abeer Sharma · Team Schizoid Rentoid
Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
This report critically examines three Phase 0 AI governance proposals from A Narrow Path, aimed at preventing artificial superintelligence (ASI) development for 20 years. It evaluates Policy 2 (Prohibit AIs capable of breaking out of their environment), Policy 5 (Licensing regime & general intelligence restrictions), and Policy 6 (International treaty on AI development redlines). Using threat modeling, historical analogies, and implementation feasibility analyses, the report identifies key vulnerabilities including ambiguous terms (e.g., "environment"), challenges regulating open-source or covert AI projects, and rigid licensing thresholds that risk stifling innovation. Historical regulatory precedents (Clean Air Act, FDA, nuclear licensing, WMD treaties) highlight crucial gaps such as the absence of explicit insurance or compensation frameworks and risks of regulatory capture. Recommendations include clarifying policy definitions, extending coverage to human-enabled breaches, employing dynamic licensing criteria, integrating incentive structures (insurance requirements, liability funds), and adopting polycentric international coordination. These enhancements aim to fortify policy effectiveness, maintaining the strategic goal of halting uncontrolled ASI development.
Reviews
Criticisms mostly didn't address the problem of ensuring that superintelligence isn't built for 20 years, but rather other concerns such as innovation, civil liberties, etc
Cite this project
@misc{sharma2025challenges,
title = {{Challenges regulating open source or convert AI projects, and rigid licensing thresholds that risk stifling innovation}},
author = {Abeer Sharma},
year = {2025},
month = jun,
note = {Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-9syc}},
url = {https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-9syc}
}More from Red Teaming A Narrow Path: ControlAI Policy Sprint
- View project: Treaty Enforcement in China
Treaty Enforcement in China
JackAI
This report red-teams A Narrow Path’s international treaty proposal by stress-testing its assumptions in the Chinese context. It identifies key failure modes—regulatory capture, compute-based loopholes, and covert …
- View project: Four Paths to Failure: Red Teaming ASI Governance
Four Paths to Failure: Red Teaming ASI Governance
Shoggoth Prevention Squad
We stress‑tested A Narrow Path Phase 0—the proposed 20‑year moratorium on training artificial super‑intelligence (ASI)—during a one‑day red‑teaming hackathon. Drawing on rapid literature reviews, historical analogues …
- View project: Moratorium on the development of general AI systems
Moratorium on the development of general AI systems
G_control
All six policies are red teamed step-by-step systematically. We initially corrected vague definitions and also found that the policies regarding the capabilities of AI systems lack technical soundness and that more …