1
$\begingroup$

I am in the process of creating one well-rounded forecast, and in my research I found a few mentionings approving the use of several forecasts combined into one. I really like this idea but I have some hesitations and questions before I apply it as a legitimate technique and I would really appreciate some insight!

1) I feel as if the combination approach loses validity when it is the same forecaster creating the different models to be combined...

2) Is it more valuable to use different types of models in the component forecasts (ie. multiple regression + ARIMAX + judgemental forecast + etc.) vs. different inputs to the same type of model (different covariates for each ARIMAX model, for example)?

3) Are there criteria or best practices that determines which accuracy measure to use across forecasts when you combine them?

Any insight is appreciated, so thank you in advance!

$\endgroup$
3
$\begingroup$

See my response below:

1) I feel as if the combination approach loses validity when it is the same forecaster creating the different models to be combined...

  • The above statement is not accurate. Research has shown that combination of forecast is more accurate than individual forecast. See this article for comprehensive article on combining forecast from the book Principles of Forecasting. (Highly recommended book) I know sometimes the links will be broken, so below is summary and conclusion of the article.

enter image description here

Any forecast loses validity if you infuse personal biases into the process be it alone or when in group. By the way forecasting is the most political function anywhere (Government, business, for profit and not for profit , and you name it)!. See another article on some useful principles on forecasting aptly called golden rule of forecasting.

2) Is it more valuable to use different types of models in the component forecasts (ie. multiple regression + ARIMAX + judgemental forecast + etc.) vs. different inputs to the same type of model (different covariates for each ARIMAX model, for example)?

  • As highlighted in the above article: Use different methods or data or both. Combining forecast is beneficial if you combine diverse set of methods so different methods are able to capture variety of information in your data. Example: Combining typically improves accuracy if you combine ARIMA and Exponential smoothing as opposed to combining two exponential or two arima models.
  • There has been variety of combining methods proposed since the seminal work of Bates and Granger, however as you will read from the above reference article, it is very hard to beat simple average on individual forecast.

3) Are there criteria or best practices that determines which accuracy measure to use across forecasts when you combine them?

  • The best accuracy measure is mean absolute error or root mean squared error.
  • MEan absolute percentage error or symmetric Mean Absolute Percent Error is another good error measure. See this exceptionally well written article by Rob Hyndman (see below on the guidance on using forecast error metrics from the above article).

enter image description here

$\endgroup$
  • 1
    $\begingroup$ I should clarify that my first bullet point was just an innate feeling not something I was saying is true, but you summed it up perfectly by saying anytime personal biases make their way into forecasts they can become comprimised. And thank you for your insight to the second point, that was how my intuition was guiding me...thank you again! $\endgroup$ – brewcrew071 Jun 16 '15 at 12:41
  • $\begingroup$ Perfect, glad to be of help $\endgroup$ – forecaster Jun 16 '15 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.