The Anatomy of a Forecast Question

What goes into a forecast question? This article outlines the key elements of a forecast question and walks you through an example question, so you can examine and learn from each component. It also discusses the different types of questions (e.g., binary and multiple choice) that forecasters are likely to see on a forecasting platform.

I. Key elements of a forecast question

It’s important to review all critical elements of a forecast question before beginning your own research. There are four key elements that we discuss here: the question itself, possible answers, background information, and question time frame. In the next section, we will show an example forecast question and explain individual components in further detail.

1. The Question: This is an objective and verifiable question for the forecaster to evaluate, which includes a clear timeframe which the event in question must occur by.
2. Possible Answers: These answer options represent every possible outcome. These options do not overlap, and should account for all extreme and edge cases.
3. Background Information: This expands and becomes visible when you click on “see more details” under each forecast question. Background information will include:

1. Definitions of any ambiguous terms.

2. Background information to familiarize forecasters with the topic and provide them with starting points for their research.

3. Resolution criteria to state the conditions by which the outcome will be judged.

4. Sources which will be relied upon for the final resolution.

4. Question starting and closing dates: The question start date is the date the question was launched and gives forecasters a sense of how long the question has been on the site. The question closing date is the last possible date that forecasts will be accepted on a question before it resolves. A question might be resolved prior to its closing date if the event in question occurs earlier. 

II. An example: analyzing insights from forecast question components

Breaking down a specific question’s components often reveals much about the topic and answer options, even before beginning your own research. In the example below, we’ve underlined key components of a forecast question and outlined our takeaways from each part.

• Blue: The first term (e.g., who, what, when, where, why) of a forecast question can tell you about the possible answers before you even look at the options. For example, “Will” here likely indicates that you are looking for a yes/no binary resolution.

• Red: This is the “subject” of the question, pointing to whose actions are in focus.

• Green: This is the objective and verifiable metric you are analyzing.

• Purple: This is specifying context related to the objective metric you are analyzing. This information tells you how to limit your research, and, in this case, why these regulatory actions could matter.

• Yellow: The time frame tells you how to limit your research and forecast accordingly.

• Pink circles: Pay special attention to the background information and resolution criteria found under “See more details” and the question closing date to know how often you should review and update your forecast, and when the question will inevitably resolve.

• Black circle: If you have doubts or questions about any component of the forecast question, submit a New Clarification request found under the settings gear icon. 

III. Types of forecast questions

It’s also helpful to know about the types of forecast questions and answer styles on a Cultivate-run forecasting site – you may have already noticed there are several! See descriptions and examples of the types used most often below:

• Binomial a.k.a. Binary: These are yes or no questions, e.g., “Will Switzerland win more medals than Austria at the Summer Olympics this year?”. These questions tend to receive the most forecasts!
• Multinomial, Single Outcome: These questions have multiple possible answers, but only one answer can occur, e.g., “Who will win the 50m dash?”
• Multinomial, Ordinal: These questions have multiple possible answers where only one answer can be correct and the answers have a natural ordering, such as numeric ranges, e.g., “How many athletes will be disqualified from the Olympics for illegal supplements?” Answer options: 0-9, 10-19, 20 or more.
• Multinomial, Multi Outcome: These questions have multiple possible answers, and several answers could occur, e.g., “Which countries will earn a medal in basketball?”
• Continuous: These questions are binary questions that are used to ask about the probability of an outcome occurring within a specified future time horizon (e.g., the next X months). The question can run in perpetuity while providing an up to date consensus forecast that the outcome will occur during the specified time horizon, e.g., “Will China invade, blockade, or attack Taiwan in the next six months?”
• Multi-Time Period: These questions elicit forecasts and 90% confidence intervals for a question over a sequence of one or more time periods, e.g., “What will revenue be in Q1? Q2? Q3? Q4?”

Our question development teams work to design each forecast question so that every element is intentional and aligns with the needs of the stakeholder or decision-maker. However, don’t hesitate to reach out for a clarification if there is any doubt about any particular component. Thanks for forecasting with us!