The AI Forecasting Hackathon

Winners for AI Forecasting Hacathon:

Congratulations to all our winners! The projects were incredible and we're excited to see what you build next

• 1st Place ($1000):
  • AI Incidents Forecasting
• 2nd Place ($500):
  • AI Shared Socioeconomic Pathways
• 3rd Place ($300):
  • AI Treaty Momentum Index (ATMI)
• 4th Place ($100):
  • ExogenousAI
• 5th Place ($100):
  • System Dynamics Game-Theoretic Model of the AI Development Race

—————————————————————————————————————————————————

The trajectory of AI development represents one of the most consequential questions for humanity's future. Understanding when and how transformative AI capabilities will emerge is critical for policy, safety research, and societal preparedness. Yet current forecasting methods struggle with unprecedented technological shifts, compounding uncertainties, and the challenge of predicting emergent capabilities.

That's why we're launching the AI Forecasting Hacthakon.

In this hackathon, you can:

• Build forecasting models and evaluation pipelines to anticipate AI capabilities and timelines
• Create tools for scenario exploration, uncertainty quantification, and model benchmarking
• Develop monitoring systems for key indicators of AI progress across research, industry, and policy
• Write policy briefs and governance proposals grounded in forecasting insights
• Explore new methodologies inspired by projects like AI 2027 and EpochAI's empirical forecasting work
• Pursue other projects that advance the field of AI forecasting!

You will work in teams over one weekend and submit open-source forecasting models, benchmark suites, scenario analyses, policy briefs, or empirical studies that advance our understanding of AI development timelines and trajectories.

💰 $2,000 in prizes will be awarded to the top projects.

Winning projects will be published openly, shared with safety researchers, and invited to continue development within the Apart Fellowship.

Tracks

1. AI Capability Forecasting & Timeline Models

This track emphasizes transparent, reproducible methods to predict AI development milestones and anticipate transformative AI capabilities. Projects may:

• Design benchmarks and empirical models to forecast transformative AI timelines and capability emergence
• Build evaluation pipelines that interpret scaling laws and compute trends (e.g., inspired by EpochAI's Direct Approach)
• Develop quantitative models predicting automation milestones for coding, research, and economic tasks
• Compare forecasting approaches (expert surveys, trend extrapolation, biological anchors) and their policy implications
• Create tools for estimating timelines to AGI using multiple methodological frameworks

2. Scenario Planning & Uncertainty Analysis

This track focuses on modeling possible AI futures, their uncertainties, and acceleration dynamics. Projects may:

• Create interactive frameworks for multi-year scenarios, referencing approaches like AI 2027's pathway modeling
• Build tools for uncertainty quantification, confidence calibration, and sensitivity analysis
• Develop "what-if" simulators exploring policy interventions and their effects on AI trajectories
• Design tabletop exercises and wargaming tools for strategic planning around critical AI decision points
• Analyze branching points and their implications for policymakers using forecasting-driven insights

3. AI Progress Monitoring & Early Warning Systems

This track focuses on real-time tracking and rapid response to AI advancement indicators. Projects may:

• Build dashboards monitoring key metrics across compute investment, model performance, and economic impact
• Create detection systems for capability jumps, breakthrough papers, or acceleration signals
• Develop APIs aggregating progress indicators from research labs, industry, and academia
• Prototype early-warning systems enabling rapid regulatory or institutional responses
• Design tools tracking algorithmic efficiency gains and their implications for timeline estimates

4. Governance, Policy & Meta-Forecasting

This track addresses integrating forecasts into governance while ensuring forecasting rigor. Projects may:

• Map sources of error and uncertainty in AI timelines (parameter choices, trend shifts, domain boundaries)
• Compare strengths/limitations of different forecasting methodologies through systematic critique
• Create educational resources ensuring rigorous, transparent forecasting practices across the field

What you will do

Participants will:

• Form teams or join existing groups.
• Develop projects over an intensive hackathon weekend.
• Submit open-source forecasting models, scenario analyses, monitoring tools, or empirical research advancing our understanding of AI trajectories

What happens next

Winning and promising projects will be:

• Awarded with $2,000 worth of prizes in cash.
• Published openly for the community.
• Invited to continue development within the Apart Fellowship.
• Shared with relevant safety researchers.

Why join?

• Impact: Your work may directly inform AI governance decisions and help society prepare for transformative AI
• Mentorship: Expert forecasters, AI researchers, and policy practitioners will guide projects throughout the hackathon
• Community: Collaborate with peers from across the globe working to understand AI's trajectory and implications
• Visibility: Top projects will be featured on Apart Research's platforms and connected to follow-up opportunities

Start here: Foundations of AI Forecasting and Timeline Research

• AI 2027 Scenario - Comprehensive scenario forecasting AI agents by 2026 and potential superintelligence by late 2027
• Epoch AI Research Portal - Interactive models and data on compute trends, algorithmic progress, and economic impacts
• Biological Anchors Framework - Ajeya Cotra's method for forecasting transformative AI using computational comparisons to biological systems
• LessWrong AI Timelines Investigation - Comprehensive survey of timeline predictions with median estimates falling within 10-40 years
• Metaculus AI Progress Tournament - Platform for crowdsourced AI forecasting
• METR Task Horizon Research - Shows AI task completion ability doubling every 7 months over past 6 years

Track 1: AI Capability Forecasting & Timeline Models:

• Epoch's Direct Approach - Compute-centric forecasting using scaling laws to directly predict performance improvements
• AI 2027 Timelines Forecast - Model predicting superhuman coding capabilities with 2027 as peak probability year
• RE-Bench Paper - METR's benchmark comparing AI agents to human experts on ML research engineering tasks
• Can AI Scaling Continue Through 2030? - Analysis predicting 2e29 FLOP training runs feasible by 2030
• Compute Trends Dashboard - Key statistics on AI trajectory including compute, costs, data, and hardware

Track 2: Scenario Planning & Uncertainty Analysis:

• AI 2027 Interactive Scenario - Full branching narrative with "slowdown" vs "race" endings
• GATE Economic Model - Epoch's compute-centric model of AI automation and economic effects
• AI Futures Project Tabletop Exercises - Over 30 iterations of AGI development simulations
• Forecasting Thread: AI Timelines - Community probability distributions
• Biological Anchors Sensitivity Analysis - Ajeya Cotra's updated forecasts with ~15% probability of TAI by 2036

Track 3: AI Progress Monitoring & Early Warning Systems:

• METR's Autonomy Evaluations - Evaluation protocols measuring AI agent performance on multi-hour tasks
• Metaculus AI Forecasting Benchmark - Tournament benchmarking AI forecasting bots against human predictions with $30K quarterly prizes
• Epoch's ML Models Database - Largest public database of notable ML models
• RE-Bench GitHub Repository - Open-source evaluation environments for AI R&D capabilities
• Forecasting Tools Framework - Python framework for building Metaculus tournament bots with benchmarking utilities

Track 4: Governance, Policy & Meta-Forecasting:

• Literature Review of TAI Timelines - Comparison of quantitative models and expert forecasts
• Biological Anchors Critiques Collection - Summaries and rebuttals in one place
• AI 2027 Security Forecast - Analysis of governance challenges including model theft risks and power concentration
• Epoch 2040 Forecasts - Predictions of coding automation, 10% GDP growth, and wild uncertainty after 2035
• AXRP Biological Anchors Interview - Discussion of forecasting methodology and parameter choices

Other possible directions and open problems

This hackathon encourages bold, original contributions. Beyond the listed tracks, participants may explore the following research directions, each presenting open challenges at the intersection of AI forecasting and strategic planning:

• What new benchmarks capture genuine progress toward transformative AI without being gameable?
• How can we better quantify uncertainty in scaling law extrapolations?
• What early indicators distinguish temporary plateaus from fundamental limits?
• How should governance adapt as forecasted timelines shorten?
• Can we develop forecasting methods robust to paradigm shifts in AI development?
• What economic models best capture AI-driven feedback loops in R&D?

🏆 Judging Criteria

1. Forecasting Relevance: Does the project advance AI timeline prediction?
   • Builds on or challenges existing forecasting literature
   • Offers novel methodologies or empirical insights
   • Focuses clearly on measurable AI progress indicators
2. Strategic Value: Does this meaningfully inform AI governance and preparedness?
   • Advances understanding of capability emergence or scaling dynamics
   • Reduces uncertainty around critical milestones
   • Has potential for real-world application in policy or research planning
3. Execution Quality: Is the project rigorous, reproducible, and well-scoped?
   • Methodologically sound with clear uncertainty quantification
   • Reproducible with documented data and assumptions
   • Designed for continued iteration or practical deployment

Top teams will receive mentorship, visibility, and the chance to continue their work through Apart Research Fellowship.

Submission Requirements

All projects must be submitted by the deadline through the official submission portal.

Your submission must include:

• A completed project report using the provided template (mandatory)
• Link to a public GitHub repository with your code (recommended)
• A brief (3-5 minute) video demonstration of your solution (optional)
• An appendix documenting any AI/LLM prompts used in your project for reproducibility (optional)

Important: Include an appendix called "Security Considerations" that outlines potential limitations of your approach and suggestions for future improvements.

❓ Frequently Asked Questions

About the Forecasting Hackathon

Q: Who can participate?
A: Anyone with a strong interest in AI safety, Forecasting, or global security. This includes researchers, engineers, students, policy analysts, and domain experts in other domains.

Q: Do I need Forecasting experience?
A: No. We provide starter resources and mentors.

Q: Is the hackathon remote?
A: Yes. Global and virtual. Talks are streamed. Collaboration happens on the Apart community Discord

Q: What should I bring to the hackathon?
A: Bring your laptop and charger. All other resources will be provided.

Q: How do teams work?
A: Teams can have up to 5 members. You can form teams in advance or join the team-matching session at the beginning of the event. Solo participants are welcome, though collaboration is encouraged.

Q: What computing resources will be available?
A: Each team will receive $400 in cloud computing credits.*

Q: Is there a code of conduct?
A: Yes. All participants must adhere to the hackathon code of conduct, which promotes responsible research, ethical AI development, and respectful collaboration.

*Cloud compute access to A100s or stronger GPUs is not available to participants from countries with active U.S. sanctions. A list of sanctioned countries can be found here.