Case study: Improving government forecasts

container-van

Our client, a government department with an existing futures team, wanted to improve their forward range and predictive accuracy with a custom training from Maby. Our plan had the following steps:

  1. A detailed consultation with the client lead to identify key risks requiring accurate forecasts.
  2. Pre-training assessment of each team member’s scope sensitivity and calibration, key attributes for accurate forecasting.
  3. A training workshop focused on forecasting the key risk topics, using each risk area as the jumping off point to explore more advanced forecasting techniques - beginning with assigning probabilities to binary yes/no event questions, then multiple outcomes, and finally exploring the construction of probability ranges and conditional forecasts. Each technique is explored using techniques to aggregate and share information among the team.
  4. At each stage of the workshop, forecasts and written comments are recorded from all team members.
  5. We provide use of our forecast aggregation software for one year following the workshop to use as an internal tool for the team.
  6. As the forecasts resolve over time, team members receive individual feedback on their forecast accuracy, allowing them to identify lessons learned and improve their next set of estimates.

Outcome

Team members:

  • Have a better understanding of their strengths and areas to target for improvement.
  • Are upskilled in generating probabilities for scenarios.
  • Have learned techniques to maximise accuracy as a team, balancing individual and collective estimates.
  • Can promptly and confidently deliver probability estimates for a range of policy choices by decision makers.

Decision makers:

  1. Have a better understanding of their futures team’s accuracy and calibration.
  2. Have a strong internal resource to draw on for fast and accurate estimates of risks and outcomes when facing decisions under uncertainty.