Maby

Forecasting Expert vs. Disease Experts

Researchers from UMass have been collecting COVID forecasts from over 30 disease experts from insitutions like Harvard and Johns Hopkins. Tom has been forecasting the same questions in real time. The results below show all the questions, forecasts, and accuracy scores.

So far, we have been more accurate than the experts!

Click here to see how the Brier scores were calculated.

How to improve your team’s forecasts in 15 minutes

We teach your team how to implement the Delphi method using our app to make measurably more accurate forecasts. The Delphi method is a proven technique developed by the Rand Corporation in the 1950s. The basic steps are:

1. Everyone in a group submits their anonymous forecasts to a question - so there is no group think or anchoring.
2. Next everyone gets to see what their teammates wrote (anonymized), along with the group's initial median forecast.
3. Participants can vote on comments they find insightful. Because comments are anonymized, voting is based on the merit of the idea, not personal politics.
4. Everyone can update their forecasts if they read compelling comments by others. An updated group median is calculated.
5. When the outcome of questions are known, then forecasts can be scored. Everyone can learn from the feedback and measure if the group's median forecast improved after updating.

Using our app to gather, share, and track forecasts, the above process can be completed in as little as 15 minutes for a question. While it would take 20 minutes for each member of a 20 person team to talk for a minute, our app allows everyone's comment to be read in less time - so meetings are shorter. By keeping score, you can measure the change in your team's forecast accuracy.

Key highlights of our 90 minute online sessions include:

• Before the meeting we convert your key issues into 3-4 forecasting questions and load them onto our app.
• During the meeting with your team, we guide your team through the Delphi method in which everyone first forecasts anonymously, then sees what everyone else thinks, votes for the best comments, and updates their forecasts.
• We expect the Delphi method to lead to measurably better forecasts (as measured by Brier score) in as little as 15 minutes per question.
• At the end of the meeting we’ll produce a special report containing all the forecasts, underlying data, estimates and top comments along with feedback and calibration information for everyone who participates.

Our goal is to make your team self-sufficient, so you can use the Delphi method and keep score even when we're not around.

Whether you need one-off support for urgent critical projections or are looking to build up key forecasting skills within your organisation, Maby can help!