Ensemble time series model

Question

I need to automate time-series forecasting, and I don't know in advance the features of those series (seasonality, trend, noise, etc).

My aim is not to get the best possible model for each series, but to avoid pretty bad models. In other words, to get small errors every time is not a problem, but to get big errors once in a while is.

I thought I could achieve this by combining models calculated with different techniques.

That is, although ARIMA would be the best approach for a specific series, it may not be the best for another series; the same for exponential smoothing.

However, if I combine one model from each technique, even if one model isn't so good, the other will bring the estimate closer to the real value.

It is well-known that ARIMA works better for long-term well-behaved series, while exponential smoothing stands out with short-term noisy series.

• My idea is to combine models generated from both techniques in order to get more robust forecasts, does it make sense?

There might be many ways to combine those models.

• If this is a good approach, how should I combine them?

A simple mean of forecasts is an option, but maybe I could get better predictions if I weight the mean according to some goodness measure of the model.

• What would be the treatment of the variance when combining models?

Answer 1

Combining forecasts is an excellent idea. (I think it is not an exaggeration to say that this is one of the few things academic forecasters agree on.)

I happen to have written a paper a while back looking at different ways to weight forecasts in combining them: http://www.sciencedirect.com/science/article/pii/S0169207010001032 Basically, using (Akaike) weights did not consistently improve combinations over simple or trimmed/winsorized means or medians, so I personally would think twice before implementing a complex procedure that may not yield a definite benefit (recall, though, that combinations consistently outperformed selection single methods by information criteria). This may depend on the particular time series you have, of course.

I looked at combining prediction intervals in the paper above, but not at combining variance as such. I seem to recall a paper not long back in the IJF with this focus, so you may want to search for "combining" or "combination" through back issues of the IJF.

A few other papers that have looked at combining forecasts are here (from 1989, but a review) and here and here (also looks at densities) and here and here. Many of these note that it is still poorly understood why forecast combinations frequently outperform single selected models. The second-to-last paper is on the M3 forecasting competition; one of their main findings was (number (3) on p. 458) that "The accuracy of the combination of various methods outperforms, on average, the specific methods being combined and does well in comparison with other methods." The last of these papers finds that combinations do not necessarily perform better than single models, but that they can considerably reduce the risk of catastrophic failure (which is one of your goals). More literature should readily be found in the International Journal of Forecasting, the Journal of Forecasting and for more specific applications in the econometrics or supply chain literature.

Answer 2

Why not specify it further? I don't think that any one model you would produce could be better or good enough than a specific choice.

With that said, if you can narrow down your choices a bit to those you can test for, and the data input can be standardized, then why not write an automated testing procedure in R?

Say you decide your data will fall within a range to be estimated by five models as well as one "fallback". Say you can characterize the input by different tests. Then just go ahead and write an R (or a program like that) algorithm that runs this for you. This works if you could produce a flowchart of which model to run based on test data, that is if any point of the decision tree is binary.

If this is not an option because the decision may not be binary, I suggest you implement a rating system based on applicable tests and run some "extreme cases" simulated data through your grid to see if the results are what you are looking for.

You can combine these things obviously, for example testing for non-stationarity may be give a definitve yes-no, while other attributes may fall into a range such as multicollinearity.
You can draw this out on paper first, then build it, simulate it with known distributions you expect to have.

Then just run the R program everytime new data arrives. I see no need to combine several models with the computational capabilities you most likely have at hand.

Answer 3

There is a nice and simple formulae for combining two forecasting methods, you just weight them multipling the first by a and the other by (1 - a), where a is found by minimizing the variance of this combined forecast. As you know the errors of both forecasting methods, you can calculate the errors of the combination wich will depend on "a". The calculation is simple when the mean of each method is = 0. For combining more than 2 methods the formulae is still "simple" in the sense that you can calculate it analytically "by hand", or also use the Solver option from EXCEL