8
$\begingroup$

It seems that OLS residuals autocorrelation is not always an issue, depending on the problem at hand. But why residuals autocorrelation would affect the coefficient standard errors? From the Wikipedia article on autocorrelation:

While it does not bias the OLS coefficient estimates, the standard errors tend to be underestimated (and the t-scores overestimated) when the autocorrelations of the errors at low lags are positive.

$\endgroup$
  • 1
    $\begingroup$ Consider an extreme case of correlation. Suppose all the errors were perfectly positively correlated. In other words, somebody had generated a single random number and added it to all the response values. How certain would you be of (say) the intercept in the regression? Would you have any clues at all concerning the size of the random value that was added? $\endgroup$ – whuber Sep 6 '14 at 22:55
  • $\begingroup$ Yes, but that is true of any missing predictor that could explain 99% of the variance and we just ignore. Why are making a specific case for $Y_{t-1}$? $\endgroup$ – Robert Kubrick Sep 6 '14 at 22:59
  • $\begingroup$ My example is not missing any predictors at all: it is only positing an extreme case of autocorrelation among the residuals. $\endgroup$ – whuber Sep 7 '14 at 13:34
  • $\begingroup$ ok, but how is this different than the case where we don't have any residuals autocorrelation, but we're not including another critical predictor? We can draw the same confidence conclusions because of that other critical predictor. The only difference is that in the case of a missing $Y_{t-1}$ critical predictor we can observe autocorrelation in the residuals. Other than that, I'm not clear what makes autocorrelation stand apart from other critical predictors. $\endgroup$ – Robert Kubrick Sep 7 '14 at 15:35
  • $\begingroup$ I'm afraid I cannot make any sense of that last comment. It's not even clear what you mean by "$Y_{t-1}$". If by that you mean a lagged variable in a time series, then you are no longer talking about OLS regression. $\endgroup$ – whuber Sep 7 '14 at 17:23
9
$\begingroup$

Suppose your OLS regression is well specified and contains all the right explanatory variables, but you have an unspecified correlation structure of the residuals: $$ y_t = x_t' \beta + \epsilon_t, \mathbb{V}[\mathbf{\epsilon}]=\Omega $$ The OLS estimates are $$ \hat\beta = (X'X)^{-1} X'Y = \beta + (X'X)^{-1} X'\mathbf{\epsilon} $$ and their variance is $$ \mathbb{V}[\hat\beta] = \mathbb{E} [ (X'X)^{-1} X'\mathbf{\epsilon}\mathbf{\epsilon}'X (X'X)^{-1} ] $$ Typically, at this stage, we'd have to assume something like existence of the probability limit of $\frac1T (X'X) \to \Sigma$, so that $$ T \mathbb{V}[\hat\beta] \to \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\mathbf{\epsilon}\mathbf{\epsilon}'X \bigr] \Sigma^{-1} = \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\Omega X \bigr] \Sigma^{-1} $$ This expression is different from what the naive OLS standard errors produce, and so in general the OLS standard errors are wrong.

Of course, if $X$ can be considered fixed, then there is no need for asymptotic approximations, and $X$ can be carried through the expectations, so that $$ \mathbb{V}[\hat\beta] = (X'X)^{-1} X'\Omega X (X'X)^{-1} $$ to the same effect.

$\endgroup$
  • 1
    $\begingroup$ All I get from this is that the higher the residuals, the higher $V[\hat{\beta}]$. So that will increase the SE, not underestimate it. And how is this specific to $Y_{t-1}$? $\endgroup$ – Robert Kubrick Sep 6 '14 at 23:06
  • $\begingroup$ Take the last formula by @StasK and consider the univariate case with nonzero off-diagonal elements (autocorrelation). See what happens in the numerator and compare that to what you get in the numerator when you assume $\Omega=\sigma^2 I$. $\endgroup$ – ekvall Sep 7 '14 at 0:20
  • $\begingroup$ @KarlOskar I still don't see anything specific to $Y_{-1}$ in the equation above, only the residuals impact the results (which is true in any case, not just $Y$ autocorrelation). $\endgroup$ – Robert Kubrick Sep 7 '14 at 16:47
  • 3
    $\begingroup$ @RobertKubrick that's exactly the point: having any unmodeled correlation is bad, autocorrelation is just a common special case of having unmodeled correlation. $\endgroup$ – Aniko Sep 7 '14 at 22:03
  • 1
    $\begingroup$ @Aniko thank you to confirm! That's what I was trying to understand in my question comments as well. $\endgroup$ – Robert Kubrick Sep 7 '14 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.