Skip to content
Apart Research
Apart Research
Contact us
Sprint projectJun 13, 2025Hythe, UK

Four Paths to Failure: Red Teaming ASI Governance

Luca De Leo, Zoé Roy-Stang, Heramb Podar, Damin Curtis, Vishakha Agrawal, Ben Smyth · Team Shoggoth Prevention Squad

Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Four Paths to Failure: Red Teaming ASI Governance

Open in a new tab (opens in new tab)
Share

We stress‑tested A Narrow Path Phase 0—the proposed 20‑year moratorium on training artificial super‑intelligence (ASI)—during a one‑day red‑teaming hackathon. Drawing on rapid literature reviews, historical analogues (nuclear, bioweapon, cryptography, and export‑control regimes), and rough‑order cost modelling, we examined four cornerstone safeguards: datacentre compute caps, training‑licence thresholds, use‑ban triggers, and 12‑day breakout‑time detection logic. Our analysis surfaced four realistic circumvention routes that an adversary could pursue while remaining nominally compliant: Jurisdictional arbitrage via non‑signatory “AI‑haven” states.

Distributed consumer‑GPU swarms operating below per‑node licence limits.

Covert runs on classified state super­computers hidden behind national‑security carve‑outs.

Offshore proxy datacentres protected by host‑state sovereignty and inspection vetoes.

Each path could power GPT‑4‑scale training within 6–18 months, undermining the moratorium’s objectives. To close these gaps, we propose ten mutually reinforcing amendments, including universal cryptographic “compute passports,” quarterly‑updating compute thresholds, an International AI Safety Commission with secondary‑sanctions powers, HSM‑bound weights, kill‑switch performance bonds, whistle‑blower bounties, whole‑network swarm detection, conditional infrastructure aid, and a 30‑month sunset‑and‑iteration clause. Implemented as a single package, these measures transform Phase 0 from a jurisdiction‑bounded freeze into a verifiable, adaptive global regime that blocks every breach path identified while permitting licensed low‑risk research.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

Does the analysis realistically assess what government agencies, resources, and expertise would be needed to implement these policies? Are the identified implementation challenges specific and grounded in understanding of how similar policies have worked (or failed) in practice? Does the submission adequately consider bureaucratic, technical, and coordination complexities involved in enforcement? How well does the analysis account for real-world constraints like budget limitations, regulatory capture, and inter-agency coordination?

Does the analysis identify specific ways the policies could fail to prevent ASI development or be circumvented by determined actors? How thoroughly does the submission examine edge cases, loopholes, or unintended consequences that could undermine the 20-year goal? Does the assessment consider different threat models (state actors, rogue researchers, corporate actors) and how policies address each? Are the identified failure modes realistic and significant, or primarily theoretical edge cases?

Does the submission cite relevant historical examples of similar policies (nuclear non-proliferation, export controls, dual-use technology regulation) to support its arguments? Are claims backed by empirical data, documented case studies, or credible expert analysis rather than speculation? How well does the analysis draw lessons from comparable regulatory domains to assess likely outcomes? Does the submission avoid making unsupported assertions about what "would" or "could" happen without evidence?

  1. First off, I appreciate the explicit choice to say " We deliberately favoured substance critiques—those that treat the

    policies as flawlessly implemented—and set aside feasibility or implementation questions". This is the heart of the exercise! Thank you.

    The discussion of regulatory flight is particularly thoughtful, especially with regards to the specific arguments about timeframes for moving to other countries and examples (we were already familiar with the Soviet bioweapons and Pakistani nuclear program examples and think they are some of the strongest bits of evidence against A Narrow Path, but thinking of FTX as an example of this is particularly inspired.)

    I think the datacentre discussion is also good, though doesn't wrestle with the core question readers are left with: why aren't players in the space doing this already, and how quickly would the optimization pressure of our polices move the companies to overcome the (real but not insurmountable) technical hurdles. The compute passports idea in this context is intriguing, however. The discussion of other ways to identify sufficient criminal suspicion for a search (e.g., power usage or heat emissions, which is in line with how many jurisdictions detect e.g. unlawful marijuana grow houses today) is also thoughtful and better-articulated than most examples I've seen.

    The offshore discussion is good, though it immediately made me think of examples of rendition during the US Global War on Terror that I wish you had brought in as well.

    The 10 ideas are quite good as well; some of these mirror our proposals (e.g., the IASC inspections) but the specifics of how to make them work are just really sharp.

    Overall, this hits the mark of giving me specific additional things that 1. I need to think about to find things to patch and 2. specific additional sentences that I can already use in discussions with policymakers to answer a "how might this work" question, driven by examples.

    Read full reviewShow less

  2. Great paper! I have reservations about adding sunset provisions.

Cite this project

@misc{leo2025four,
  title = {{Four Paths to Failure: Red Teaming ASI Governance}},
  author = {Luca De Leo and Zoé Roy-Stang and Heramb Podar and Damin Curtis and Vishakha Agrawal and Ben Smyth},
  year = {2025},
  month = jun,
  note = {Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/four-paths-to-failure-red-teaming-asi-governance-se53}},
  url = {https://apartresearch.com/sprints/projects/four-paths-to-failure-red-teaming-asi-governance-se53}
}

More from Red Teaming A Narrow Path: ControlAI Policy Sprint

All 20 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026