More accurate forecasts. Empirically.

Make better estimates about the future with training, facilitation, software, support and feedback.

Contact us

More accurate than disease experts

Researchers from UMass have been collecting COVID forecasts from over 30 disease experts from insitutions like Harvard and Johns Hopkins. We've been forecasting along side and keeping score. Our scoreboard below shows all the questions, forecasts, and accuracy scores.

So far, we have been more accurate than the experts!

Forecast
(10%-90% interval)
Accuracy Score
(aka Brier score)
wk Question TOM EXPERTS Outcome TOM EXPERTS
AVERAGE BRIER SCORE
(green is better, red is worse)
0.165 0.235
5/10 How many confirmed US cases will there be on May 17? (in millions) 1.48
(1.45-1.57)
1.5
(1.47-1.6)
1.48 0.062 0.135
5/10 How many cases per day will Washington state average for the week ending June 7? 170
(30-500)
331
(158-644)
5/10 How many deaths will Pennsylvania report as of June 13? (in thousands) 7.6
(5.2-12)
7.2
(5.2-10)
5/3 What will the 7 day average new cases be in Texas for the week ending June 13? 1150
(400-3000)
1350
(750-2300)
5/3 How many US COVID deaths will occur in 2020? (in thousands) 160
(90-800)
256
(118-1212)
5/3 How many confirmed US cases will there be on May 10? (in millions) 1.33
(1.27-1.39)
1.35
(1.25-1.44)
1.33 0.066 0.129
5/3 How many US states/territories will report more new cases for September compared to June? 7
(1-42)
21
(2-45)
4/26 How many confirmed US cases will there be on May 3? (in millions) 1.14
(1.09-1.21)
1.14
(1.07-1.24)
1.152 0.174 0.195
4/26 When will weekly new US deaths first fall below 5,000 (Sunday to Saturday)? 5/23
(5/9-6/26)
6/13
(5/16-7/4)
4/26 What will the average new daily cases be for Georgia between 5/10 and 5/16 inclusive? 900
(150-5000)
1044
(579-2292)
659 0.159 0.173
4/19 As of May 9, how many COVID-19 related deaths will have occurred in Illinois? (in thousands) 2.4
(1.7-4.8)
2.8
(1.8-5.2)
3.3 0.294 0.170
4/19 As of May 9, how many COVID-19 related deaths will have occurred in Louisiana? (in thousands) 1.85
(1.55-3.4)
2.8
(1.85-5.0)
2.2 0.129 0.170
4/19 How many confirmed US cases will there be on April 26? (in thousands) 930
(870-990)
950
(860-1060)
959 0.222 0.127
4/19 How many deaths due to COVID-19 will occur in the US in 2020? (in thousands) 80
(48-750)
150
(72-517)
4/19 As of May 9, how many COVID-19 related deaths will have occurred in the US? (in thousands) 56
(47-77)
70
(51-106)
73 0.420 0.170
4/12 How many US COVID-19 deaths will occur by June 1? (in thousands) 71
(48-127)
71
(45-131)
4/12 When will deaths peak in the US? April
(April-June)
April
(March-July)
4/12 How many US COVID-19 deaths will occur by May 1? (in thousands) 50
(36-66)
46
(34-68)
59 0.111 0.180
4/12 How many states will report more than 1000 deaths by May 1? 12
(9-16)
9
(4-15)
14 0.193 0.712
4/12 How many US COVID-19 cases will there be on April 19? (in thousands) 755
(710-795)
780
(690-905)
751 0.093 0.151
4/5 When will US COVID-19 hospitalizations peak? April
(April-June)
May
(April-August)
4/5 How many US COVID-19 cases will there be on April 12? (in thousands) 520
(470-680)
670
(510-900)
551 0.103 0.336
3/29 Will NY state report fewer than 1,000 new cases on the day of April 28? 28%
(-)
50%
(-)
no 0.157 0.500
3/29 How many US COVID-19 cases will there be on April 5? (in thousands) 410
(250-590)
420
(260-580)
332 0.123 0.145
3/29 When will US hospitalizations peak? April
(April-June)
May
(April-August)
3/29 How many deaths due to COVID-19 will occur in the US in 2020? (in thousands) 240
(75-1200)
260
(80-1000)
3/22 Will US COVID-19 deaths exceed 245,500 by the end of 2020? (percent probability YES) 45%
(-)
50%
(-)
3/22 Will there will be a second wave of US cases in 2020? (US active cases declines by 50% and subsequently increases by 50% AND 2nd wave must be at least 50% as high) (probability YES) 52%
(-)
73%
(-)

Scoreboard explainer: The questions have a few different formats, including 10/90 confidence intervals and multi-bin questions. In order to keep score, we have converted the format to a form that allows us to keep score. When a question asks for 10/50/90 percentiles, we have used the percentile forecasts provided by the experts as cut points for a multi-bin question. By definition the experts' forecasts to that multi-bin question are 10% / 40% / 40% / 10%. We have assigned probabilities to those bins consistent with Tom's 10/50/90. This is done as close to real time as possible and posted on twitter. In cases where the question is originally a multi-bin question, we have approximated what the forecaster's 10/50/90 with simple interpolation and presented the results in the table above.


Online Forecasting Facilitation

Goal: Have your group make measurably better forecasts on the issues that matter most to you during a 90 minute online meeting.

Key highlights:

  • Before the meeting we convert your key issues into 3-4 forecasting questions and load them onto our app.
  • During the meeting with your team, we guide your team through the Delphi method in which everyone first forecasts anonymously, then sees what everyone else thinks, votes for the best comments, and updates their forecasts.
  • We expect the Delphi method to lead to measurably better forecasts (as measured by Brier score) in as little as 15 minutes per question.
  • After the meeting you receive the forecast data, feedback, and calibration information for everyone who participates.
  • We view ourselves as coaches to get the most out of your team. Like personal trainers, we aim to teach you how to forecast effectively yourself, but we can train your team as frequently as you like.

Whether you need help improving your forecast accuracy or want to outsource your foresight entirely to us, Maby can help!