According to Armstrong there is ample empirical evidence that dampening trends in uncertain and complex long term forecasting helps improve accuracy/reduce forecasting errors. What I'm not able to find is a formula or an implementation of dampening trends. I know there exists dampening trends in exponential smoothing which is not what I'm looking for. I was looking for a simple formula which would dampen recursively the forecasted values based on a user provided dampening factor.

Lets suppose I have the following dataset,first 8 values are actual and the remainder are forecast, how would you dampen the forecasted values with the user provided dampening factors that dampens the trend recursively i.e., as the time progresses there should be more dampening as the uncertainits increase in long run?

I have artificially provided dampening values and dampened the forecasted values, is there a mechanical process formula to create the dampening value ?

Any suggestions/recommendations of articles/formula spreadsheet implementation of dampening trends are greatly appreciated.

Best Regards

enter image description here

Edit: I would like to clarify what I'm trying to achive in terms of an illustrative example.

Lets suppose if we have a trend forecast as shown below,

    stl.ap <-stl(AirPassengers,s.window="periodic")

## extract Trend component

    trend.ap <- stl.ap$time.series[,2]

 ## Ignore Seasonal data since this is an example to determine
 ## how to dampen a trend

linear.ap <- tslm(trend.ap~+ trend + I(trend^2))
forecast.ap <- forecast(linear.ap, h=36)


enter image description here

As can be seen above, this is an unrealistic projection of trend, is there a formula that recursively dampens this trend. I would like to dampen the trend heavily as the time horizon increases.

  • 2
    $\begingroup$ When you say "dampen the forecasted trend", there has to be a forecast trend to dampen, but you have offered no forecasting approach to dampen. Some models naturally dampen trends, of course, but there are many which do, and the way your question is phrased it sounds like you have a model in mind which you'd like to alter so that its present forecast trends are dampened. $\endgroup$ – Glen_b Jul 8 '14 at 5:45
  • 2
    $\begingroup$ Again, you're saying 'dampen the forecast' as if you already have one, but without specifying a model to try to apply some kind of dampening to. Are you seeking to have some mechanical process that simply 'flattens out' any sequence of numbers? I doubt that's what Armstrong intends. The usual way one would dampen forecasts would depend on the form of the model - you'd rewrite the functional form to 'dampen' it, but exactly what that means depends on how the model works. $\endgroup$ – Glen_b Jul 8 '14 at 14:36
  • 2
    $\begingroup$ Then you need to reread my comments above which explain why such an approach will tend to be unsatisfactory; the necessary characteristics will tend to be different with different forecasting models. $\endgroup$ – Glen_b Jul 8 '14 at 15:26
  • 1
    $\begingroup$ That quote doesn't establish they did what you are asking for (i.e. something applied only to any arbitrary set of numbers rather than to a method). It only establishes that dampening tends to help (which was not in dispute). If you specify say exponential smoothing, it's easier to see the appropriate way to dampen it. Same with Theta-sm - it starts with particular kinds of smoothing and then dampens those. To my understanding the two methods don't use exactly the same dampening, but ones tailored to their assumptions. I'm simply asking for the same kind of information they'd have had. $\endgroup$ – Glen_b Jul 8 '14 at 17:14
  • 2
    $\begingroup$ "I'm looking is a simple formula that would recursively dampened the trend like the exponential smoothing dampened trend method." -- again, yes, as soon as you tell me the method. Dampening forecasts is not one size fits all, as I have explained before. What's suitable for ordinary exponential smoothing is not necessarily suitable for triple- exponential smoothing. What's suitable for seasonal series is not suitable for random walks, and so on. Give some kind of framework, and we can get somewhere. $\endgroup$ – Glen_b Jul 8 '14 at 17:16

Dampening can be thought of as a special case of shrinkage methods; these methods as a whole tend to reduce uncertainty in estimates (yet another circumstance of trading bias for variance, an ever-recurring theme in statistics, though in some cases, such as many involving variable-selection, shrinkage can reduce both bias and variance).

There are many methods by which forecasts are produced, but they have a wide variety of characteristics.

Consider, for example, a case where we just observe independent random values with a constant mean. Then dampening would involve moving our forecasts a little toward zero from the sample mean.

In methods that estimate a local-mean effect element (some form of adaptive level, say), dampening could move the forecast of that recent level toward 0 (or, for slightly different models, toward the overall mean), so that forecast trends tended to 'fall back' from the most recent excursion over longer forecast horizons.

In methods that have an element of recent local linear trend (some form of adaptive trend, say), dampening could move the forecast of that recent trend toward 0, so that forecast trends tended to 'flatten out' over longer forecast horizons.

Now consider a case where, for example, there's seasonal effects around some other trend - dampening of the seasonal component would tend to shrink the forecast strength of the seasonality, "smoothing out" the wiggle over longer forecast horizons.

The text by Hyndman and Athana­sopou­los, Forecasting principles and practice, (freely available on-line as well as in dead-tree form) has a section on dampening, but you need some of the preceding sections for context - the models in sec 7.4 are damped versions of the models in sec 7.2, Holt linear trend models). I highly recommend investigating this book. [You may find the older book on forecasting by Makridakis, Wheelwright and Hyndman in libraries or on forecaster's bookshelves. It's also very handy.]

That section of text on dampening includes something like the kind of thing you're asking for - a tunable smoothing parameter, $\phi$, between $0$ and $1$, to produce a dampening effect ... but as you'll see if you look at that section, it's of a different form for different models - the additive trend and multiplicative trend models use different formulas!

So how might we "dampen" a linear least squares fit*?

*(Note that I don't particularly regard this as a suitable approach for very many forecasting problems, but the Hyndman and Athana­sopou­los text is a better place to investigate the merits of various approaches. Nevertheless, let us proceed, since a discussion of the issues covers many of the issues one must consider when trying to dampen models more generally)

A linear regression doesn't have any 'local' component to it at all, it's a global model. In any case, as I mentioned at the start, dampening can be thought of as shrinkage, and we can still shrink that estimate of the linear component toward 0.

But we then have the question of about which center do we 'tilt' the line? The obvious approach for an actually global model would be to do it about the mean:

The line in center-slope form would be $y-\bar{y} = \beta (x-\bar{x})+\varepsilon_t$.

The point forecasts are $\hat{y_t} = \bar{y} + b (x_t-\bar{x})$, where $b=\hat{\beta}$, the slope estimate.

If $T$ is the last observed time, then the point forecasts are

$\hat{y_{T+k}} = \bar{y} + b (x_{T+k}-\bar{x})$.

One simple and commonly used dampening is to multiply it at each step by a constant $\phi$, where $0<\phi<1$ which would let the slope shrink with time:

$\hat{y_{T+k}} = \bar{y} + b_{T+k} (x_{T+k}-\bar{x})$,

where $b_{T+k} = \phi b_{T+k-1} = \phi^k b_{T} = \phi^k b$.

enter image description here

However, as I said, a global model doesn't necessarily make as much sense as some other options if it's a situation in which we'd want to apply such dampening; we might consider instead pivoting the slope around a point somewhere near the end (nearer to $t=T$), to make it more local... but if we think the model should be local in that sense, we should probably be looking at locally-linear models to start with, and then dampen those trends.

There's some distinct similarities in general approach I took here to the one used for the additive trend model, though that's a local model, not a global one - and thereby more suited to forecasting trends that tend to be linear for a while, but where the linear trend isn't constant in the long term.

That approach of progressive multiplication of a model-component by $\phi$ could be used for any number of models if applied to the parts it makes sense to shrink to be smaller - it could be applied to seasonal components, or AR parameters, or any number of other things. Different components of forecasting models may even be shrunk at different rates.

[That progressive multiplication by a constant (geometric shrinkage) isn't the only way to shrink components of a model, but it's the most common.]

  • $\begingroup$ Excellent. Thank You for taking effort in answering my question. I do have both the forecasting books in Hard copy that you recommended. I personally still favor the Makridakis et al a classic, simply because of exceptional writing style. I also like the latest Hyndman et al because of thr R codes accompanying the book. $\endgroup$ – forecaster Jul 10 '14 at 19:49
  • $\begingroup$ It is also sad that none of the books talk about dampening trends in detail, they have a cursory mention of it in exponential smoothing context and not go in depth on why it works. Armstrong's book and article is also an eye opener for a practitioner like me. $\endgroup$ – forecaster Jul 10 '14 at 19:51
  • $\begingroup$ I also dont know of any software that implements the seasonal factor dampening as outlined in this article forecastingprinciples.com/paperpdf/Damped_Seasonality_Intro.pdf based on the work of miller and williams, still no software gives the open to do something like this sciencedirect.com/science/article/pii/S0169207002000778, this article by the way was awarded the best paper in 2003 by their peers, still no major software vendor has used procedures eve though we know that they work empirically. $\endgroup$ – forecaster Jul 10 '14 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.