Moratorium on the development of general AI systems
Gideon Daniel Giftson T · Team G_control
Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.
All six policies are red teamed step-by-step systematically. We initially corrected vague definitions and also found that the policies regarding the capabilities of AI systems lack technical soundness and that more incentives are needed to entice states to sign the treaty. Further, we discover a lack of equity in the licensing framework, and a lack of planning for black-swan events. We propose an oversight framework right from the manufacturing process of silicon chips. We also propose calling for a moratorium on the development of general AI systems until the existing tools for analyzing them can catch up. Following these recommendations still won't guarantee the prevention of ASI for 20 years, but ensures that the world is on track to even tackle such a system if it is somehow created.
Reviews
Thoughtful analysis, points out a number of potential issues such as sandbagging, and the lack of incentives to join the treaty system as formulated in Phase 0
Cite this project
@misc{t2025moratorium,
title = {{Moratorium on the development of general AI systems}},
author = {Gideon Daniel Giftson T},
year = {2025},
month = jun,
note = {Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-fjwu}},
url = {https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-fjwu}
}More from Red Teaming A Narrow Path: ControlAI Policy Sprint
- View project: Treaty Enforcement in China
Treaty Enforcement in China
JackAI
This report red-teams A Narrow Path’s international treaty proposal by stress-testing its assumptions in the Chinese context. It identifies key failure modes—regulatory capture, compute-based loopholes, and covert …
- View project: Four Paths to Failure: Red Teaming ASI Governance
Four Paths to Failure: Red Teaming ASI Governance
Shoggoth Prevention Squad
We stress‑tested A Narrow Path Phase 0—the proposed 20‑year moratorium on training artificial super‑intelligence (ASI)—during a one‑day red‑teaming hackathon. Drawing on rapid literature reviews, historical analogues …
- View project: Red Teaming A Narrow Path: ControlAI Policy Sprint by Aritra Das and Vaani Goenka
Red Teaming A Narrow Path: ControlAI Policy Sprint by Aritra Das and Vaani Goenka
Controllers Of Ai
This research analyses two proposed AI governance policies – prohibiting recursive self-improvement in AI systems and mandating safety cases for deployment – through historical precedent analysis, agent-based modeling, …