Skip to content
Sprint projectJun 13, 2025Forest City, Malaysia

Challenges regulating open source or convert AI projects, and rigid licensing thresholds that risk stifling innovation

Abeer Sharma · Team Schizoid Rentoid

Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Challenges regulating open source or convert AI projects, and rigid licensing thresholds that risk stifling innovation

Share

This report critically examines three Phase 0 AI governance proposals from A Narrow Path, aimed at preventing artificial superintelligence (ASI) development for 20 years. It evaluates Policy 2 (Prohibit AIs capable of breaking out of their environment), Policy 5 (Licensing regime & general intelligence restrictions), and Policy 6 (International treaty on AI development redlines). Using threat modeling, historical analogies, and implementation feasibility analyses, the report identifies key vulnerabilities including ambiguous terms (e.g., "environment"), challenges regulating open-source or covert AI projects, and rigid licensing thresholds that risk stifling innovation. Historical regulatory precedents (Clean Air Act, FDA, nuclear licensing, WMD treaties) highlight crucial gaps such as the absence of explicit insurance or compensation frameworks and risks of regulatory capture. Recommendations include clarifying policy definitions, extending coverage to human-enabled breaches, employing dynamic licensing criteria, integrating incentive structures (insurance requirements, liability funds), and adopting polycentric international coordination. These enhancements aim to fortify policy effectiveness, maintaining the strategic goal of halting uncontrolled ASI development.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

Does the analysis realistically assess what government agencies, resources, and expertise would be needed to implement these policies? Are the identified implementation challenges specific and grounded in understanding of how similar policies have worked (or failed) in practice? Does the submission adequately consider bureaucratic, technical, and coordination complexities involved in enforcement? How well does the analysis account for real-world constraints like budget limitations, regulatory capture, and inter-agency coordination?

Does the analysis identify specific ways the policies could fail to prevent ASI development or be circumvented by determined actors? How thoroughly does the submission examine edge cases, loopholes, or unintended consequences that could undermine the 20-year goal? Does the assessment consider different threat models (state actors, rogue researchers, corporate actors) and how policies address each? Are the identified failure modes realistic and significant, or primarily theoretical edge cases?

Does the submission cite relevant historical examples of similar policies (nuclear non-proliferation, export controls, dual-use technology regulation) to support its arguments? Are claims backed by empirical data, documented case studies, or credible expert analysis rather than speculation? How well does the analysis draw lessons from comparable regulatory domains to assess likely outcomes? Does the submission avoid making unsupported assertions about what "would" or "could" happen without evidence?

  1. Criticisms mostly didn't address the problem of ensuring that superintelligence isn't built for 20 years, but rather other concerns such as innovation, civil liberties, etc

Cite this project

@misc{sharma2025challenges,
  title = {{Challenges regulating open source or convert AI projects, and rigid licensing thresholds that risk stifling innovation}},
  author = {Abeer Sharma},
  year = {2025},
  month = jun,
  note = {Submitted to Red Teaming A Narrow Path: ControlAI Policy Sprint, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-9syc}},
  url = {https://apartresearch.com/sprints/projects/red-teaming-a-narrow-path-controlai-policy-sprint-9syc}
}

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026