9
$\begingroup$

It seems that OLS residuals autocorrelation is not always an issue, depending on the problem at hand. But why residuals autocorrelation would affect the coefficient standard errors? From the Wikipedia article on autocorrelation:

While it does not bias the OLS coefficient estimates, the standard errors tend to be underestimated (and the t-scores overestimated) when the autocorrelations of the errors at low lags are positive.

$\endgroup$
6
  • 2
    $\begingroup$ Consider an extreme case of correlation. Suppose all the errors were perfectly positively correlated. In other words, somebody had generated a single random number and added it to all the response values. How certain would you be of (say) the intercept in the regression? Would you have any clues at all concerning the size of the random value that was added? $\endgroup$
    – whuber
    Sep 6, 2014 at 22:55
  • $\begingroup$ Yes, but that is true of any missing predictor that could explain 99% of the variance and we just ignore. Why are making a specific case for $Y_{t-1}$? $\endgroup$ Sep 6, 2014 at 22:59
  • $\begingroup$ My example is not missing any predictors at all: it is only positing an extreme case of autocorrelation among the residuals. $\endgroup$
    – whuber
    Sep 7, 2014 at 13:34
  • $\begingroup$ ok, but how is this different than the case where we don't have any residuals autocorrelation, but we're not including another critical predictor? We can draw the same confidence conclusions because of that other critical predictor. The only difference is that in the case of a missing $Y_{t-1}$ critical predictor we can observe autocorrelation in the residuals. Other than that, I'm not clear what makes autocorrelation stand apart from other critical predictors. $\endgroup$ Sep 7, 2014 at 15:35
  • $\begingroup$ I'm afraid I cannot make any sense of that last comment. It's not even clear what you mean by "$Y_{t-1}$". If by that you mean a lagged variable in a time series, then you are no longer talking about OLS regression. $\endgroup$
    – whuber
    Sep 7, 2014 at 17:23

2 Answers 2

10
$\begingroup$

Suppose your OLS regression is well specified and contains all the right explanatory variables, but you have an unspecified correlation structure of the residuals: $$ y_t = x_t' \beta + \epsilon_t, \mathbb{V}[\mathbf{\epsilon}]=\Omega $$ The OLS estimates are $$ \hat\beta = (X'X)^{-1} X'Y = \beta + (X'X)^{-1} X'\mathbf{\epsilon} $$ and their variance is $$ \mathbb{V}[\hat\beta] = \mathbb{E} [ (X'X)^{-1} X'\mathbf{\epsilon}\mathbf{\epsilon}'X (X'X)^{-1} ] $$ Typically, at this stage, we'd have to assume something like existence of the probability limit of $\frac1T (X'X) \to \Sigma$, so that $$ T \mathbb{V}[\hat\beta] \to \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\mathbf{\epsilon}\mathbf{\epsilon}'X \bigr] \Sigma^{-1} = \Sigma^{-1} {\rm plim} \bigl[ \frac1T X'\Omega X \bigr] \Sigma^{-1} $$ This expression is different from what the naive OLS standard errors produce, and so in general the OLS standard errors are wrong.

Of course, if $X$ can be considered fixed, then there is no need for asymptotic approximations, and $X$ can be carried through the expectations, so that $$ \mathbb{V}[\hat\beta] = (X'X)^{-1} X'\Omega X (X'X)^{-1} $$ to the same effect.

$\endgroup$
6
  • 1
    $\begingroup$ All I get from this is that the higher the residuals, the higher $V[\hat{\beta}]$. So that will increase the SE, not underestimate it. And how is this specific to $Y_{t-1}$? $\endgroup$ Sep 6, 2014 at 23:06
  • $\begingroup$ Take the last formula by @StasK and consider the univariate case with nonzero off-diagonal elements (autocorrelation). See what happens in the numerator and compare that to what you get in the numerator when you assume $\Omega=\sigma^2 I$. $\endgroup$
    – KOE
    Sep 7, 2014 at 0:20
  • $\begingroup$ @KarlOskar I still don't see anything specific to $Y_{-1}$ in the equation above, only the residuals impact the results (which is true in any case, not just $Y$ autocorrelation). $\endgroup$ Sep 7, 2014 at 16:47
  • 3
    $\begingroup$ @RobertKubrick that's exactly the point: having any unmodeled correlation is bad, autocorrelation is just a common special case of having unmodeled correlation. $\endgroup$
    – Aniko
    Sep 7, 2014 at 22:03
  • 1
    $\begingroup$ @Aniko thank you to confirm! That's what I was trying to understand in my question comments as well. $\endgroup$ Sep 7, 2014 at 22:06
0
$\begingroup$

Auto correlation (Exogeneity) : If the observation of residual in t1 is dependent on the observation of residual in t0, then it basically violates the underlying assumption of OLS which states "Error term is independently distributed and not correlated". This can bias the estimation of Beta coefficient of OLS term.

When such trend is observed in the residuals, the residuals of regression model picks up the influence of those variables that are affecting the dependent variables that aren't part of the regression equation.

The persistence in excluded variables is most of the times a cause for autocorrelation. This is more prevalent in Time series data.

This can be mitigated by using some of the transformations techniques like,

  1. Paris Winston Transformation of Data
  2. Differencing the dependent variables
  3. Differencing all variables

One simpler approach would be,

  1. estimating the linear model using OLS.
  2. Compute the residuals.
  3. Regress the residuals of all Independent variables and lagged variables.
  4. Use t-test, if the coefficient of lagged residual is significant, we can reject the null of independent errors.

Test to detect Auto correlation: **

  1. Durbin - Watson
  2. Bruesch - Godfrey

This assumption/issue is violated/exempted if the explanatory variables are stochastic, for instance when they are measured with error, or are endogenous.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.