Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm just starting to look into exponential smoothing models. Is there a way to fit a linear regression with exponentially-smoothed errors, similar to the standard technique of fitting a regression with ARMA errors? Or is this generally considered a bad idea?

Of course, I can come up with heuristics, like fit with AR(1) errors, then exp-smooth the residuals (possibly on the differences, then look for a trend) but that feels very ad hoc.

share|improve this question
Like you said very very very ad hoc "like fit with AR(1) errors, then exp-smooth the residuals (possibly on the differences, then look for a trend) but that feels very ad hoc." and very probably just wrong ! – IrishStat Jun 4 '12 at 22:18
up vote 1 down vote accepted

What you want to do is to investigate Transfer Function Models ( also known as Dynamic Regression ) which seamlessly integrates "regression" and "arima" . If you were to assume that the arima portion was a simple exponential smoothing model AND the regression component to be a simple contemporaneous relationship you would have your objective. This of course could be silly as you are assuming the model structure. In general you would want to identify the best of these components making sure that pulses, level shifts , seasonal pulses and local time trends were included when necessary. Additionally care should be taken to ensure that the model parameters and errror variance are "proven" to be constant over time. If you had some real data we could take this discussion further.

share|improve this answer
Thanks, that gives some food for thought. Do you have any good links describing transfer function models? My current idea is to put together a state-space model (West&Harrison- or Durbin&Koopman-style), plug in a diffuse prior, no state variance (except on the constant and trend), and turn the crank, but that method tends to depend a lot on the exact variances chosen. – Johann Hibschman Jun 4 '12 at 18:26
you might want to look at… and others that I have commented on .…… – IrishStat Jun 4 '12 at 19:33
more: If you form the Transfer Function Model y(t)=W(B)*X(t)+[THETA(B)/PHI(B)]*a(t) the operator [THETA(B)/PHI(B)] is the "smoothing component". For examnple if PHI(B)=1.0 and THETA(B)=1-.5B this would imply a set of weights of .5,.25,.125,... . in this way you could provide the answer to optimizing the "weighted moving linear regression" rather than assuming it's form. From… – IrishStat Jun 4 '12 at 20:18
upon reflection perhaps PHI(B) should be [1-B] or the y and x be the first order differences of the observed Y and X – IrishStat Jun 4 '12 at 20:36

If you're thinking mainly about an EW version of an MA model, I'm not sure why this would be necessary. In general MA models are more of a pain to estimate since they rely on MLE. Oftentimes, people would just throw in sufficient lags in an AR model to cover the MA structure. If the worry is about fitting a particular PACF structure, then it might be appropriate to estimate fractionally integrated models.

share|improve this answer
What I want is a regression setting a baseline, then looking for a slow-moving error trend on top of that. That's hard to turn into a straight AR or MA model. GARCH seems like a solution in search of a problem here, as usual; I'm not concerned about getting the volatility right, just the baseline. – Johann Hibschman Jun 4 '12 at 18:33
I think ARIFMA (fractional integration) might suit your needs then. That type of model is meant to handle long memory in the conditional mean. No need for the snark, but you weren't particularly clear in your question. – John Jun 4 '12 at 19:09
Thanks, I'll look into ARIFMA. Not trying to be snarky, but I'm a bit of a GARCH-skeptic. Probably comes from mostly looking at monthly data, rather than daily or intraday, where improved vol modeling doesn't pay off. Mortgages, housing, and econometrics, oh my. – Johann Hibschman Jun 4 '12 at 19:35
In my experience, it depends on the application. I've used Garch with daily, but intraday gets tricky. – John Jun 4 '12 at 19:47

It is perfectly reasonable to model the residuals of a regression model as a stationary ARMA model. This is done with harmonic regression. Exponential smoothing is IMA(1,1). So it would imply nonstationary residuals from the regression.

share|improve this answer
The obvious problem with this is that it is a two step approach rather than a simultaneous estimation ala a Transfer Function , but that's just my opinion. – IrishStat Jun 4 '12 at 20:24
@IrishStat I would disagree if the residual model were stationary. However, since exponential smoothing is nonstationary and the mean drifts the trend in the residual might have better been handled in the regression piece. – Michael Chernick Jun 4 '12 at 20:33

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.