My objective is to find the best ARIMA or Exponential Smoothing model to forecast one time serie. I know that, to choose the best model, one can make tests to know if the ts has trend, seasonality, is stationary, etc and then choose consequently the appropriate model.
But, what about doing backtesting with all possible models and choose the best one? i.e. the one that has better accuracy over time?
Let's put an example. Suppose that, for a given ts, we have available monthly data from Jan2016 to this last month (Oct2018) and we want to forecast the next November. The strategy is: fit all the ARIMA models (a representative subset) and all the Exponential Smoothing models for the last n months, calculate the accuracy and choose, for forecast November, the one with the best overall accuracy.
I mean, fit all the models with Jan16-Feb18 and forecast Mar18, calculate accuracy, then fit all the models with Jan16-Mar18 and forecast Apr18, calculate accuracy, etc.
With the actual power of the computers, this can be done in seconds.
What do you think?