Forecasting Autonomous AI Bio-Threat Design Capabilities: Six Models Converge on 2031

The topic is very relevant, and I like that you consider different modes of forecasting. However, at its current state, I have low confidence in the forecasts produced by this project.

Documentation leaves room to be improved. For example, it’s left unclear why Model 2 and Model 4 were excluded. For Model 5, it’s unclear to me what the different experts forecasted. Model 6 doesn’t really do scenario analysis despite its name - it just multiplies some numbers together.

The project could have benefitted from engaging more deeply with existing literature on the topic. Rather than coming up with a “standardized test” yourself, you could have used an existing benchmark, such as the Virology Capabilities Test, and forecasted performance by models on that benchmark. That would’ve been a more feasible project given the time limit.

I think forecasting biological capabilities is an important question! However, I think there are quite a few implementation decisions that were left out that seem somewhat arbitrary. For example, why were Models 2 and 4 excluded from the weighting? How were the weights for the other models chosen? For the Expert Consensus model, are the estimates across the different organisations comparable (ie are they trying to estimate the same thing)? How was the 100-point framework created, why are the four categories weighted equally, and why do you consider 75 points as a critical threshold?

The more complex the approach is, the more decision points and assumptions you will have to justify. You should consider simplifying the whole approach, or reduce the scope of your research question.