7
$\begingroup$

I'm still trying to expand my statistics and forecasting technique knowledge.

Right now I'm forecasting seasonal contact patterns, so the simplest model I can understand with seasonality is a Holt-Winters/ Triple exponential smoothing model.

For many of our decisions, we not only need predictions for one month out, but also two months out, and even as far as 6 months out, or more.

Obviously accuracy increasingly goes out the window as you try to forecast 6 months out, but, I guess we do what we can.

The first thing I immediately noticed after creating the triple exponential smoothing model and initiated the seasonal/ trend values -- is that there are certainly different 'optimal' alpha (level), beta (trend) and gamma (seasonality) values, depending on whether you are forecasting one month out, or 6 months out.

As you might have guessed, the 1-month forecast has far greater accuracy for a test data set with a higher alpha value (stronger recent values) -- and the 6 month is better with a lower alpha value.

Not that it matters much, but the 2 month is closer to the 1 month model, and the 3-4-5 seems to be closer to the 6 month, but they all have their local 'optimal' values.

So if I need a rolling forecast for the next 6 months --- am I to use one model, or am I supposed to use a different set of alpha-beta-gamma values depending on how far out the predicted value is?

I have a minor background in statistics, but I know it's easy to misuse/ misread them. I don't know if using two or three different models like this can lead to mistakes.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
4
$\begingroup$

What you are talking about is using a "direct" forecasting strategy rather than the more popular "recursive" forecasting strategy.

In a recursive strategy, one model is fitted, usually based on minimizing the one-step forecast mean squared error, and the forecasts for all future horizons are estimated by iterating the equations over time. If the model is linear, and the same as the data generating process, this is optimal. But in reality, the model is at best an approximation to whatever generated the data, and most real-world phenomena are non-linear.

Contrary to what @Tom Reilly asserts, there is considerable academic literature on this problem, but very little software implements alternatives to the recursive approach. The problem is addressed, and some of the associated literature is referenced, in my recent paper with Souhaib Ben Taieb: https://robjhyndman.com/publications/boostingar/

The bottom line is that you should try different approaches and choose what works best for your problem. In this case, I suggest you try the direct approach (with different parameters for different horizons) and compare the forecast accuracy with what you get using the recursive approach.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the help Rob. I suppose I do not often look back at these questions, but I will definitely take a look at your paper, and am glad to hear there is literature on this and I'm not taking crazy pills. I think at the time, I did eventually find an article on this topic: sciencedirect.com/science/article/pii/S0169207002000109 -I'm not an expert on the topic by any means, so I can't debate the merits, but at least it seemed to imply that multiple models may be better in some cases. I of course have more reading to do. $\endgroup$
    – John Babson
    Commented Oct 20, 2014 at 15:54
  • $\begingroup$ The link to paper is broken, it resides here now: robjhyndman.com/papers/rectify.pdf $\endgroup$
    – user285512
    Commented Jul 28, 2020 at 19:57
  • $\begingroup$ Actually, that's a different paper. I've updated the link in the answer with the correct paper. $\endgroup$
    – Rob Hyndman
    Commented Jul 28, 2020 at 23:10
1
$\begingroup$

This is a very common issue in business forecasting, where we want sensible forecasts for the short term without sacrificing long term performance.

You can test your data to see this by going back over multiple lags and holdout periods to see which models work best at different lags-- maybe one model does well 1-3 periods out, but another does much better over a longer horizon.

The forecast and accuracy functions from the forecast package in R make doing this sort of thing quite straightforward.

Once you have that data, you will have some more subjective choices to make-- is using a single model that has some connection to explaining the data preferable, even if it has more error than a mixed approach that would use different models for different time horizons?

Improve this answer
$\endgroup$
-1
$\begingroup$

John,

I am not sure who is feeding you these theories about model parameters being optimal for periods out, but it just isn't right. Please share the source of your comments(books, web pages, software, etc.) I would like to read more about this. You want a model to describe the historical variations and also the periods that are outliers and then forecast it. That's it.

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ No one is feeding me these theories -- I may have come to the conclusions myself. I mean, I have a few books on forecasting, but none of them mention creating a model for multiple t+1, t+2, t+3... t+n ... periods out forecasting. The truth of the matter is, when using the test data, there may be greater accuracy (in this case MAPE) with a higher alpha value when you are forecasting one period ahead, and a lower alpha value when you are forecasting 6 periods ahead. Whether or not this has to do with overfitting the model, I'm not sure. But that's what I've found. $\endgroup$
    – John Babson
    Commented Jun 17, 2014 at 15:51
  • $\begingroup$ I'm not using an alpha 'solver' -- mostly trial and error - but I'm pretty certain the 'solver' would come up with different alpha values for the the t+1 forecast vs the t+6 forecast. For the sake of simplicity you can use one model to forecast every period, but would this not be less accurate? $\endgroup$
    – John Babson
    Commented Jun 17, 2014 at 15:55
  • $\begingroup$ Ok, I understand why you are trying to do what you are doing now, but there is no support for this in a "book" so you are "freestyling" here :). You can't use a "goal seek" approach to minimize MAPE for time series problems. The "goal" here is to think about the data and the causal variables that are driving it and then include them and the forecasts of those to drive your forecasting process all the while checking for outliers, changes in level/trend/seasonality/parameters/variance andof course making sure the residuals are random. $\endgroup$
    – Tom Reilly
    Commented Jun 18, 2014 at 5:48
  • $\begingroup$ I agree that you definitely need to include casual variables, account for outliers, etc from time series forecasts. Checking if the residuals are random of course is important - yes. But primarily, I am looking for prediction accuracy (which can be aided with casual variables, or detecting systematic bias, of course) --- I am in a business setting and forecast accuracy pretty much IS the end goal. In terms of prediction, my method may be wrong, possibly, but I cannot quite see why right now. Why are there different optimum alpha values for each horizon? Why only pick one set for all horizons? $\endgroup$
    – John Babson
    Commented Jun 18, 2014 at 14:48
  • $\begingroup$ You are using a model to go against data when you should be modeling your data. "Let the data speak". The steps are Identification, Estimation, Diagnostic Checking and then Forecasting. None of those steps entail "hey is it optimized for two periods out?" I would be happy to discuss off-line. See my profile. $\endgroup$
    – Tom Reilly
    Commented Jun 19, 2014 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.