1
$\begingroup$

Im trying to perform some time series analysis on 2015 and 2016 monthly recorded data to see what method is best for forecasting 2016 monthly values for the remainder of the year. The data has an annual cycle and I already have values for the first 9 months out of 2016. Using Winter's method, I forecasted values for the first 9 months of 2016 and summed up the absolute differences between these forecasted values and the actual 2016 values I already had. From there I calculated the 2016 monthly forecasted value to be off from the 2016 monthly actual value by a 12% average. I then tried a second method where I summed up the absolute differences between the 2015 actual monthly average 2016 actual monthly value. Long story short this elementary method resulted in predictions being off by an average of 6%. My question is how is the Winters error percentage so much higher than using a straghtforward average method. I should also note the MAPE predicted in Minitab using Winters for fitted 2015 monthly data was 2%. Can anyone make sense of all this for me?

$\endgroup$
3
  • $\begingroup$ The answer found here gives a good discussion of how surprisingly often simple forecasting methods outperform more complex ones. Particularly since your data is small (less than 2 full seasons), estimating parameters for complex models is difficult, frequently resulting in over-fitting. Add to that that Winter's method may not match your data (there are many more models just in the ETS family) and it isn't surprising at all that a mean forecast does better. $\endgroup$ – Barker Oct 26 '16 at 19:45
  • $\begingroup$ Thank you for the response. Why would Minitab also give me a MAPE of 1.59 when I my calculated my MAPE to be 12%. $\endgroup$ – Roy Young Oct 26 '16 at 20:14
  • $\begingroup$ I am not familiar with Minitab, so I am not sure how they are predicting the MAPE but the error is probably due to over-fitting. Hyndman discusses in his blog that when error estimates are made for predictions, they only consider the error term in the model (in other words the model residuals). Since you don't have a ton of data, you can get a pretty tight fit (small residuals) using a complex model. That doesn't mean the model will generalize, and if it doesn't, you end up with much higher out of sample errors than expected. $\endgroup$ – Barker Oct 26 '16 at 21:36