Combining Forecasts I am in the process of creating one well-rounded forecast, and in my research I found a few mentionings approving the use of several forecasts combined into one. I really like this idea but I have some hesitations and questions before I apply it as a legitimate technique and I would really appreciate some insight!
1) I feel as if the combination approach loses validity when it is the same forecaster creating the different models to be combined...
2) Is it more valuable to use different types of models in the component forecasts (ie. multiple regression + ARIMAX + judgemental forecast + etc.) vs. different inputs to the same type of model (different covariates for each ARIMAX model, for example)?
3) Are there criteria or best practices that determines which accuracy measure to use across forecasts when you combine them?
Any insight is appreciated, so thank you in advance!
 A: See my response  below:
1) I feel as if the combination approach loses validity when it is the same forecaster creating the different models to be combined...


*

*The above statement is not accurate. Research has shown that combination of forecast is more accurate than individual forecast. See this article for comprehensive article on combining forecast from the book Principles of Forecasting. (Highly recommended book) I know sometimes the links will be broken, so below is summary and conclusion of the article. 



Any forecast loses validity if you infuse personal biases into the process be it alone or when in group. By the way forecasting is the most political function anywhere (Government, business, for profit and not for profit , and you name it)!. See another article on some useful principles on forecasting aptly called golden rule of forecasting.
2) Is it more valuable to use different types of models in the component forecasts (ie. multiple regression + ARIMAX + judgemental forecast + etc.) vs. different inputs to the same type of model (different covariates for each ARIMAX model, for example)?


*

*As highlighted in the above article: Use  different methods or data
or both. Combining forecast is beneficial if you combine diverse set
of methods so different methods are able to capture variety of
information in your data. Example: Combining typically improves accuracy  if you combine ARIMA and Exponential smoothing as opposed to combining two exponential or two arima models.

*There has been variety of combining methods proposed since the seminal work of Bates and Granger, however as you will read from the above reference article, it is very hard to beat simple average on individual forecast.


3) Are there criteria or best practices that determines which accuracy measure to use across forecasts when you combine them?


*

*The best accuracy measure is mean absolute error or root mean squared error.

*MEan absolute percentage error or symmetric Mean Absolute Percent Error is another good error measure. See this exceptionally well written article by Rob Hyndman (see below on the guidance on using forecast error metrics from the above article).



