Improve your team's forecasts

Learn forecasting best practices. Measure your accuracy improvement.

Contact us

Forecasting Expert vs. Disease Experts

We believe that keeping score of forecasts is the key to assessing and improving your forecast accuracy.

As the covid-19 pandemic hit the United States, we were invited to submit forecast estimates alongside a CDC-funded panel of experts in public health. Our forecasts have been more accurate than virus experts despite having no training in public health, virology, or epidemiology. We simply follow the same sound forecasting techniques that we teach to our clients.

Below you can see a complete record of our coronavirus forecasts.

Forecast
(10%-90% interval)
Accuracy Score
(Brier score)
wk Question TOM EXPERTS Outcome TOM EXPERTS
AVERAGE BRIER SCORE
(green is better, red is worse)
0.194 0.247
5/10 How many cases per day will Washington state average for the week ending June 7? 170
(30-500)
331
(158-644)
155 0.289 0.713
5/10 How many confirmed US cases will there be on May 17? (in millions) 1.48
(1.45-1.57)
1.5
(1.47-1.6)
1.48 0.062 0.135
5/10 How many deaths will Pennsylvania report as of June 13? (in thousands) 7.6
(5.2-12)
7.2
(5.2-10)
6.2 0.214 0.180
5/3 What will the 7 day average new cases be in Texas for the week ending June 13? 1150
(400-3000)
1350
(750-2300)
1779 0.323 0.166
5/3 How many US COVID deaths will occur in 2020? (in thousands) 160
(90-800)
256
(118-1212)
5/3 How many confirmed US cases will there be on May 10? (in millions) 1.33
(1.27-1.39)
1.35
(1.25-1.44)
1.33 0.066 0.129
5/3 How many US states/territories will report more new cases for September compared to June? 7
(1-42)
21
(2-45)
4/26 How many confirmed US cases will there be on May 3? (in millions) 1.14
(1.09-1.21)
1.14
(1.07-1.24)
1.152 0.174 0.195
4/26 When will weekly new US deaths first fall below 5,000 (Sunday to Saturday)? 5/23
(5/9-6/26)
6/13
(5/16-7/4)
6/20 0.376 0.158
4/26 What will the average new daily cases be for Georgia between 5/10 and 5/16 inclusive? 900
(150-5000)
1044
(579-2292)
659 0.159 0.173
4/19 How many deaths due to COVID-19 will occur in the US in 2020? (in thousands) 80
(48-750)
150
(72-517)
4/19 As of May 9, how many COVID-19 related deaths will have occurred in the US? (in thousands) 56
(47-77)
70
(51-106)
73 0.420 0.170
4/19 As of May 9, how many COVID-19 related deaths will have occurred in Illinois? (in thousands) 2.4
(1.7-4.8)
2.8
(1.8-5.2)
3.3 0.294 0.170
4/19 As of May 9, how many COVID-19 related deaths will have occurred in Louisiana? (in thousands) 1.85
(1.55-3.4)
2.8
(1.85-5.0)
2.2 0.129 0.170
4/19 How many confirmed US cases will there be on April 26? (in thousands) 930
(870-990)
950
(860-1060)
959 0.222 0.127
4/12 Which of the next 6 months will see the highest total number of deaths nationwide in the US for COVID-19 illness? April
(April-June)
April
(March-July)
4/12 How many US COVID-19 deaths will occur by May 1? (in thousands) 50
(36-66)
46
(34-68)
59 0.111 0.180
4/12 How many states will report more than 1000 deaths by May 1? 12
(9-16)
9
(4-15)
14 0.193 0.712
4/12 How many US COVID-19 cases will there be on April 19? (in thousands) 755
(710-795)
780
(690-905)
751 0.093 0.151
4/12 How many US COVID-19 deaths will occur by June 1? (in thousands) 71
(48-127)
71
(45-131)
99 0.178 0.180
4/5 How many US COVID-19 cases will there be on April 12? (in thousands) 520
(470-680)
670
(510-900)
551 0.103 0.336
4/5 Which month (March-August) will see the highest total number of hospitalizations nationwide in the US for COVID-19 illness? April
(April-June)
May
(April-August)
3/29 Will NY state report fewer than 1,000 new cases on the day of April 28? 28%
(-)
50%
(-)
no 0.157 0.500
3/29 How many deaths due to COVID-19 will occur in the US in 2020? (in thousands) 240
(75-1200)
260
(80-1000)
3/29 How many US COVID-19 cases will there be on April 5? (in thousands) 410
(250-590)
420
(260-580)
332 0.123 0.145
3/29 Which month (March-August) will see the highest total number of hospitalizations nationwide in the US for COVID-19 illness? April
(April-June)
May
(April-August)
3/22 Will US COVID-19 deaths exceed 245,500 by the end of 2020? (percent probability YES) 45%
(-)
50%
(-)
3/22 Will there will be a second wave of US cases in 2020? (US active cases declines by 50% and subsequently increases by 50% AND 2nd wave must be at least 50% as high) (probability YES) 52%
(-)
73%
(-)

Click here to see how the Brier scores were calculated.


Build your team's forecasting capabilities

Maby helps your team produce accurate forecasts using our custom software, expertise in forecasting best practices, and experience making forecasting practical for busy organizations. Gain the confidence that you know the what top organizations are already doing.

calibration curve demo dot plot demo 10 50 90 demo

Goals: We’ll work with you to identify the most important estimates for your organisation. From project completion dates, key risks, quarterly outlook projections, we compile a register of required estimates.

Training: We can offer as much or as little training as you require. For some clients, building up their in-house forecasting capacity is the most important goal. Other clients prefer monthly sessions.. Either way, we have feedback and training tools to improve your team’s forecasts, and avoid the risk of common cognitive biases which can throw off estimates.

Facilitation: We’ll gather your team and walk them through the estimates they need to make- harnessing the wisdom of your crowd. Our process and software to rapidly record, anonymise, share and aggregate information. We find that 15 minutes per topic is enough to see measurable improvement in accuracy. Want an outside view into your future timelines? We can supply expert forecasters to join your team and add their insights.

Reporting: At the end of the process, you will receive a full report of every estimate, aggregated for accuracy, with qualitative commentary drawn from your team’s comments. Each participant will receive a report of their own estimates for future review, and we can check back in and update the forecasts as new information comes to light.

Feedback and improvement: For longer run projects, we can gather multiple forecasts from your team and offer detailed feedback as forecasts resolve - from calibration to resolution to noise and bias reduction, we can help your team improve their predictive accuracy over time, creating a valuable in-house resource for you to call on any time you’re facing uncertainty.

Contact us