Jun 13, 2025
Red Teaming A Narrow Path: ControlAI Policy Sprint
Abeer Sharma
This report critically examines three Phase 0 AI governance proposals from A Narrow Path, aimed at preventing artificial superintelligence (ASI) development for 20 years. It evaluates Policy 2 (Prohibit AIs capable of breaking out of their environment), Policy 5 (Licensing regime & general intelligence restrictions), and Policy 6 (International treaty on AI development redlines). Using threat modeling, historical analogies, and implementation feasibility analyses, the report identifies key vulnerabilities including ambiguous terms (e.g., "environment"), challenges regulating open-source or covert AI projects, and rigid licensing thresholds that risk stifling innovation. Historical regulatory precedents (Clean Air Act, FDA, nuclear licensing, WMD treaties) highlight crucial gaps such as the absence of explicit insurance or compensation frameworks and risks of regulatory capture. Recommendations include clarifying policy definitions, extending coverage to human-enabled breaches, employing dynamic licensing criteria, integrating incentive structures (insurance requirements, liability funds), and adopting polycentric international coordination. These enhancements aim to fortify policy effectiveness, maintaining the strategic goal of halting uncontrolled ASI development.
Tolga Bilge
Criticisms mostly didn't address the problem of ensuring that superintelligence isn't built for 20 years, but rather other concerns such as innovation, civil liberties, etc
Tolga Bilge
Well-written, and correctly identifies many of the potential downsides of A Narrow Path. If restrictions on open source AI are necessary to prevent superintelligence for 20 years, we believe this is an acceptable cost, and indeed we clearly say that models that fall under the licensing regime must not be open sourced.
Seems reasonable that Narrow Path should point to a definition of what an environment is
Some of the risks identified, such as that of regulatory capture I agree are important - and while we took measures to try to address them, not completely satisfactory.
Cite this work
@misc {
title={
@misc {
},
author={
Abeer Sharma
},
date={
6/13/25
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
