Forecasting Multi-Agent Systems
Michael Mulet, Jonas Mullet, Charlees Renshaw Willams, Nanubala Gnana Sai, Moritz Weckbecker · Team The SOARing 7 - 2
Submitted to The AI Forecasting Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
We analyze the effect Mutil-Agent Systems will have on AI capabilities and support our analysis with experimental data.
Reviews
* While not technically included in the submission, I do appreciate appendix B! I think it's critical that we not only consider the impact of different interventions (e.g. multi-agent systems), but also inspect the mechanisms for why and why not that may happen.
* Related to comparison between SAS and MAS systems, maybe one could leverage Chain-of-Thought models with equal compute budget to an MAS system as equivalent? This could be an interesting way to more fairly assess differences.
* How were the designs of the MAS systems used defined? If they are not taken from existing literature which has proven architecture and prompting strategies, then it is quite possible the conclusions found are simply the case because the MAS setup is sub-optimal.
I think the multi-agent work is interesting, and in particular when forecasting AI capabilities we often forget about multi-agent capabilities. I quite like the empirical benchmarking across different model sizes and configurations.
However, I think this seems to be primarily an evaluations project rather than a forecasting project. While the submission includes theoretical analysis about AI timelines and some forecasting questions in the appendices, the core work doesn't really seem to involve making predictions.
Cite this project
@misc{mulet2025forecasting,
title = {{Forecasting Multi-Agent Systems}},
author = {Michael Mulet and Jonas Mullet and Charlees Renshaw Willams and Nanubala Gnana Sai and Moritz Weckbecker},
year = {2025},
month = nov,
note = {Submitted to The AI Forecasting Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/forecasting-multiagent-systems-1qhn}},
url = {https://apartresearch.com/sprints/projects/forecasting-multiagent-systems-1qhn}
}More from The AI Forecasting Hackathon
- View project: System Dynamics Game-Theoretic Model of the AI Development Race
System Dynamics Game-Theoretic Model of the AI Development Race
System Dynamics BCN
A Game theoretic / System Dynamics model of the race dynamics of the US, China, and EU, as a follow up to the Armstrong et al. (2016) paper “Racing to the Precipice”. We find preliminary results where knowledge of …
- View project: ExogenousAI
ExogenousAI
Fibonacci
Current AI capability forecasting methodologies, including EpochAI's Direct Approach and Biological Anchors framework, primarily rely on internal metrics such as training compute and scaling laws while assuming stable …
- View project: AI Incidents Forecasting
AI Incidents Forecasting
KLACE
This research develops a framework for forecasting AI incidents to help predict future risks. We have developed two models that forecasts incidents which include calibrated 90% prediction intervals with backtests. These …