# Combining Forecasts: Best Information to Solicit from Forecasters?

Suppose

• Statistician $m=1$ produces a set of $h$-step-ahead point forecasts $\hat{x}_{t+h|t, 1}$ of $x_{t+h}$ where $x_{t+h} \in [0,1]$.
• Also, this point forecast could come with:
• a predictive density $f_{{x}_{t+h|t,1}}(x|x_t,...,x_1)$
• a predictive error $\hat{\sigma}_{t+h|t, 1}$
• a $(1-\alpha)\%$ predictive interval $(c_{t+h|t,1,\alpha,lb},c_{t+h|t,1,\alpha,ub})$
• or no measure of certainty in forecast at all
• After seeing this first forecast $\hat{x}_{t+h|t, 1}$, there are $M$ statistically unsophisticated forecasters with domain knowledge who could each produce their own point forecasts $\hat{x}_{t+h|t, m}$, $m=2,...,M+1$.
• After seeing all $\hat{x}_{t+h|t, m}$, $m=1,...,M+1$, a combined forecast is constructed $\bar{x}_{t+h|t, m} = \sum_{m=1}^{M+1} w_{t+h|t,m}\hat{x}_{t+h|t, m}$ where $w_{t+h|t,m} \geq 0$ and $\sum_{m=1}^{M+1}w_{t+h|t,m}=1$

Q: What additional piece of info would you solicit from the forecasters $m=2,...,M+1$ to help you construct the optimal forecast combination weights $w_{t+h|t,m}$? How would you use that info to construct the weights?

• A backtested measure of model forecast accuracy (e.g., MAPE) could exist for $m=1$ but does not exists for $m=2,...,M+1$
• These forecasters are statistically unsophisticated, so asking for their $\hat{\sigma}_{t+h|t, m}$ might not be as meaningful as asking for their predictive interval
• These forecasters don't have a lot of time and, so, are unlikely to produce predictive densities for each of their forecasts
• Forecaster $m$ usually only produces a forecast in periods $t$ where they believe the forecast $m=1$ is flawed. So $\hat{x}_{t+h|t, m}$ won't exist for all $t$

• predictive interval for the $m>1$ forecast (which, theoretically, would be constructed independently from the $\hat{x}_{t+h|t,1}$)
• probability that the $m>1$ forecast is closer to the actual than the $m=1$ forecast (i.e., $\Pr\{|\hat{x}_{t+h|t, m}-x_{t+h}| < |\hat{x}_{t+h|t, 1}-x_{t+h}|\}$ where $m>1$)
• probability that the the the actual is greater than the $m=1$ forecast (i.e., $\Pr\{x_{t+h} > \hat{x}_{t+h|t, 1}\})$