The flu burden has subsided for the season and with it the third annual FluSight Challenge has concluded. The top three prize-winners earning a share of the $5,000 prize pool are:

1. SpottedBear – $3,432
2. skmmcj – $914
3. datscilly – $297

Congratulations to the top three winners – who also earned the gold, silver, and bronze medals in the tournament – and congratulations to all the prizewinners!

Our returning partner for this project, Dr. Thomas McAndrew, who leads Lehigh University's Computational Uncertainty Lab, shared the following:

We wish to thank everyone who submitted forecasts of the peak timing and intensity of influenza hospitalizations throughout this 2023/24 season. Forecasts supported training a mechanistic model to generate forecasts for all 50 states in the country. Your contributions have led to a successful abstract submission (below), and this work has spurred two manuscripts: one focused on a computational technique to extend forecasts of peak timing/intensity from 10 states to all 50 states; and a second focused on how to train a mechanistic model on human judgment ensemble forecasts of peak timing/intensity. Your work here builds upon previous human judgment-support mechanistic modeling on a synthetic outbreak. Thank you again for all the hard work and dedication during the influenza season.

Abstract: For the 2022/2023 season, influenza accounted for 31m cases, 360k hospitalizations and 21k deaths, costing US healthcare ~$4.6 billion. Forecasts of influenza hospitalizations provide health officials with advanced warning. Prior research has revealed that forecasts generated by human judgment are similar in accuracy to mathematical models. However, little work has compared an equally weighted vs performance-weighted human judgment ensemble. We collected weekly human judgment forecasts of the peak number of incident hospitalizations (peak intensity) and the epidemic week where this peak occurs (peak time) for 10 US states from Oct 2023 to March 2024. The median, 25th, and 75th percentiles were extracted for each week. We found that, for the performance-weighted ensemble, forecast uncertainty decreased just before and after peak intensity and peak timing. However, uncertainty for the equally-weighted ensemble does not decrease. For both ensembles, the median prediction of peak intensity is smaller than the truth at the beginning of the season and approaches the truth monotonically. For peak time, the performance-weighted ensemble median prediction is later than the truth and approaches the truth only after the true peak time is observed. We have observed that the performance-weighted ensemble tends to produce more accurate forecasts of peak timing and intensity compared to an equally-weighted ensemble. One potential mechanism for this boost in accuracy was that the performance-weighted ensemble correctly assigned large weights to experienced forecasters and small weights to inexperienced forecasters. Experienced forecasters may be needed when asked to predict peak timing/intensity of an ongoing outbreak.

You can see more details and the full leaderboard for the FluSight Challenge 2023/24 here.

We will be in touch with prize winners to arrange payments. We are also happy to facilitate donations to the organizations listed here.

Congratulations again to SpottedBear, skmmcj, and datscilly!