Jun 13, 2025
Power, Proxies and People: Red-Teaming Phase 0 of A Narrow Path to Stop AI Superintelligence
Anusha Asim, Sergei Smirnov, Jackson Paulson
This project involved a red team analysis of Phase 0 of A Narrow Path, a foundational AI governance framework aimed at preventing the emergence of artificial superintelligence (ASI). The analysis critically examined five key Phase 0 policies: (1) a total ban on AI systems improving other AI systems, (2) a licensing system for advanced AI development, (3) thresholds for detecting general intelligence, (4) global governance through institutions like GUARD and IASC, and (5) the absence of behavioral influence monitoring.
Using thematic critiques, historical analogies, and a quantitative simulation of regulatory capture, the red team identified major weaknesses in both policy design and enforcement. Key vulnerabilities include the impracticality of a blanket AI-on-AI improvement ban, the risk of AI manipulating human cognition, the susceptibility of licensing regimes to corruption and monopolization, inadequate metrics for identifying general intelligence, and the marginalization of the Global South in governance structures.
The project concluded with several recommendations: adopting a risk-tiered approach to AI improvement, implementing behavioral influence audits, reforming licensing to prevent monopolies, investing in intelligence measurement science, and establishing more inclusive, globally representative institutions to coordinate AI safety efforts.
Tolga Bilge
The threat identified of AIs influencing humans is an interesting one. Seems like it would be good if we considered that more.
Funding intelligence metrology seems like a good suggestion.
Discussion of possible stifling of innovation and international power dynamics seem orthogonal to the question of how to prevent superintelligence for 20 years, and the latter is to a large extent addressed in our specific formulation of GUARD. E.g. the provision of access to the most capable safe AIs to signatory countries.
Cite this work
@misc {
title={
@misc {
},
author={
Anusha Asim, Sergei Smirnov, Jackson Paulson
},
date={
6/13/25
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
