4
$\begingroup$

Is there an intuitive mathematical explanation?

$\endgroup$
4
$\begingroup$

I'm not sure about an "intuitive mathematical" explanation, but perhaps I can provide some intuition.

From the standpoint of a typical OLS regression model, the assumptions are that the residuals (really errors) are independent and normally distributed with constant variance. These assumptions are necessary to ensure that the sampling distribution of your estimates will have the assumed form and thus that your standard errors, confidence intervals, and p-values are correct.

  1. If the variance is not constant, you have heteroscedasticity. This will mean that your SEs are larger (not smaller) than they should be. The result is that you have less statistical power than you should. The reason is that the residuals scatter more widely about the regression line in one proportion of the range than another. As a result, those data contain less information about the location of the conditional mean (the vertical position of the regression line at that point) than the model assumes. They should be down-weighted (and the region with lower residual variance should be up-weighted), but if each point gets full weight anyway, then the model thinks 'these data vary a lot, we just aren't sure where the line is supposed to be'. Thus if ordinary least squares is used to compute the estimates rather than weighted least squares, the SEs are larger with heteroscedasticity. To understand this more fully, it may help to read my answer here: Efficiency of beta estimates with heteroscedasticity.

  2. If your residuals are autocorrelated, they are not independent. Whether your SEs will be larger or smaller depends on the nature of the autocorrelation. Specifically, if you have positive autocorrelation, the SEs will be smaller than they should be; on the other hand, if you had negative autocorrelation (which never really happens in practice), you would have SEs that are larger than they should be.

    Here's a simple way to think about it: Consider estimating only the mean. If your data are positively autocorrelated, and your first datum is, say, above the mean, then the second is more likely than not to be above the mean as well. After all, the point of being positively autocorrelated is that the next data point will be close to wherever the last data point was. The result of this is that your whole sample may be equally likely to be above or below the true mean, but the data will tend to cluster together more tightly than they would if they were independent. Hence, you SEs will be smaller than they would be with independent data, but your data will actually fluctuate (from sample to sample) more widely. That is, your SEs will be too small.

    Now consider that your data are negatively autocorrelated. Again, your first datum is above the mean. But because your data are negatively autocorrelated, your next datum will be that far below the mean. Thus, even your estimate of the mean from only your first two data will tend to be very close to the true mean. This process continues as you gather more data. Your data will scatter more widely than they would under independence, but counter-intuitively the amount that your sample mean will vary from the population mean will be less than under independence. That is, your SEs will be too large.

$\endgroup$
  • 1
    $\begingroup$ Are you sure about your answer regarding what heteroskedasticity in the errors does -- specifically, that OLS-based SEs are larger than they should be if heteroskedasticity is present? At the very least, you should be more specific about what forms of heteroskedasticity you're allowing. $\endgroup$ – Mico Mar 29 '15 at 8:34
  • 1
    $\begingroup$ @gung, I share the concerns of Mico that, most of the time, the default standard errors are too small, leading to inflated t-statistics and therefore overly liberal tests. This is not a general result, though. If, for instance, $E(\epsilon_i^2) = bx^{−2}_i$, $b>0$, we may also have that standard errors are too small, leading to conservative tests. So it can go either way, I'd say. $\endgroup$ – Christoph Hanck Mar 29 '15 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.