8
$\begingroup$

I understand the AR(p) model: its input is the time series being modelled. I'm completely stuck when reading about the MA(q) model: its input is innovation or random shock as it's often formulated.

The problem is I can't imagine how to get an innovation component having no model of the (perfect) time series already (i.e. I think $\varepsilon=X_{\rm observed}-X_{\rm perfect}$, and that's probably wrong). Moreover, if we can get this innovation component in-sample, how can we get it when doing a long-term forecast (model error term as a separate additive time series component)?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

When the unobserved error terms are autocorrelated, there are at least 4 possible strategies since you can't just add the errors into your model:

  1. Use OLS with a corrected variance-covariance matrix (such as Newey-West)
  2. Transformation of the model
  3. Feasible Generalized Least Squares
  4. Instrumental Variables

(2) is probably most common. OLS and FGLS are appropriate for non-scalar residual variance matrices. IV is good when you have a regressor correlated with the error term. Transformations can be useful for both.

Prais-Winsten and Conchrane-Orcutt are common examples of (3) for first-order autocorrelation. These links will illustrate the mechanics nicely.

This post includes some real world examples. In the coupon example, you might imagine adding them as regressors if you could get the data. In the other examples, that makes less sense and (1)-(4) provide a feasible alternative.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thanks for your answer. Can you please clarify, does the above means that the MA(q) model must internally use an AR model to estimate innovation term $\hat{\varepsilon}=y-\hat{y}$? $\endgroup$
    – werediver
    Apr 24, 2014 at 8:17
  • 1
    $\begingroup$ You regress $y$ on $x$, get the residuals $\hat u$, and regress the residuals on their first lag to get $\rho$. Once you have $\rho$, you can transform the data. Does that make sense? $\endgroup$
    – dimitriy
    Apr 24, 2014 at 10:41
  • $\begingroup$ If $\rho$ is now the MA(1) parameter (assuming the "regress" is for "do simple linear regression"), then yes, it makes much sense! $\endgroup$
    – werediver
    Apr 24, 2014 at 10:56
  • $\begingroup$ That is correct. $\endgroup$
    – dimitriy
    Apr 24, 2014 at 11:07
  • $\begingroup$ This is the most straightforward explanation, thanks a lot. I've found a solution via MA(1) as AR($\infty$) representation which I can understand, but I don't deeply understand MA(1) to AR($\infty$) transformation and can't generalize the solution for the MA(q) model. I think I can generalize your explanation for the MA(q) case though (via regression of $y_t$ on ${x_{t-1},...,x_{t-p}}$ and then the residuals $\hat{u}_t$ on ${\hat{u}_{t-1},...,\hat{u}_{t-q}}$; is this too naive? whether it has to be $p=q$?). $\endgroup$
    – werediver
    Apr 24, 2014 at 11:33
4
$\begingroup$

When trying to get an intuitive real world picture of MA or AR (or ARMA or ARIMA if you are extending it) I often find it useful to think of carry over effects, that is something happening in one period carries over into the next.

Here's an example: say you are modelling newspaper sales. The noise (random error) in such a model could sensibly incorporate the relatively short lived effect of newspaper headlines while the rest of the model deals with more stable things like trend and seasonality (now I'm assuming an ARIMA model but if you want a pure MA model imagine no trend or seasonality for the paper). Although the newspaper headline effect is modelled as error we might decide that this effect does indeed carry over into the next few days (a good story brings in readers who then fade away again). This would invite the inclusion of an MA term in the model - the carryover of the effect of the previous error term into the current time period.

You can think in the same way about the AR term only what is carried over here is part of the effect of the whole of the previous days sales.

Hope that helps

Improve this answer
$\endgroup$
5
  • $\begingroup$ Well, yes... Thank you. Propogating of the effect of the error term. I think it's the concept of the MA model. But having a concrete time series $X$, what is fed into MA equation (except $\Theta$)? How we could "extract" the error term from a time series (what is assumed to be the error term? 1st order derivative? independent white noise? difference with AR model (then how to deal with it out-of-sample where there will be only AR model?)?)? One more step, please :) $\endgroup$
    – werediver
    Apr 23, 2014 at 14:23
  • $\begingroup$ Hi - If I understand you correctly you are asking a) how you fit the model to actual data (i.e. obtain an estimate of properties the error term along with estimates of the parameters) and b) how you then forecast using the model (given that no error term will be involved). Is that right? $\endgroup$
    – Simon Raper
    Apr 24, 2014 at 8:56
  • $\begingroup$ I'm not actually interested in the method of fitting the model (finding the best $\Theta$), but I'm interested in estimating the error term from a time series. I also interested in how to forecast using the model when there will be no error term (because there will be no actual data). So yes, I'd say you understand me correctly. $\endgroup$
    – werediver
    Apr 24, 2014 at 9:08
  • 2
    $\begingroup$ It's one and the same thing. The fitting of the model gives you both the coefficients and the parameters that describe the distribution of the error term. Since the error is assumed to be normally distributed with mean 0 this just means estimating the variance. Any method that fits the model (from memory is usually Yule Walker) will give you variance of the error. Forecasting with an MA model is quite interesting. Basically as you roll the model forward no more error terms are introduced and the MA forecast quickly settles down to a straight line (if the order of the MA model is relatively low $\endgroup$
    – Simon Raper
    Apr 24, 2014 at 10:03
  • $\begingroup$ It seems I've found a key: we can estimate MA(1) via its AR($\infty$) representation by expressing $\varepsilon$ from the AR equation and numerically solving $\hat\Theta=\underset{\Theta}{\arg\min}\displaystyle\sum_{t=2}^{n}(X_t-\Thetaε_{t-1})^2$ (from here here, equation (12.27)). $\endgroup$
    – werediver
    Apr 24, 2014 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.