Why do autocorrelation and heteroskedasticity under-report the sample variance of OLS estimates?

Question

Is there an intuitive mathematical explanation?

Answer 1

I'm not sure about an "intuitive mathematical" explanation, but perhaps I can provide some intuition.

From the standpoint of a typical OLS regression model, the assumptions are that the residuals (really errors) are independent and normally distributed with constant variance. These assumptions are necessary to ensure that the sampling distribution of your estimates will have the assumed form and thus that your standard errors, confidence intervals, and p-values are correct.

1. If the variance is not constant, you have heteroscedasticity. This will mean that your SEs are larger (not smaller) than they should be. The result is that you have less statistical power than you should. The reason is that the residuals scatter more widely about the regression line in one proportion of the range than another. As a result, those data contain less information about the location of the conditional mean (the vertical position of the regression line at that point) than the model assumes. They should be down-weighted (and the region with lower residual variance should be up-weighted), but if each point gets full weight anyway, then the model thinks 'these data vary a lot, we just aren't sure where the line is supposed to be'. Thus if ordinary least squares is used to compute the estimates rather than weighted least squares, the SEs are larger with heteroscedasticity. To understand this more fully, it may help to read my answer here: Efficiency of beta estimates with heteroscedasticity.

2. If your residuals are autocorrelated, they are not independent. Whether your SEs will be larger or smaller depends on the nature of the autocorrelation. Specifically, if you have positive autocorrelation, the SEs will be smaller than they should be; on the other hand, if you had negative autocorrelation (which never really happens in practice), you would have SEs that are larger than they should be.

   Here's a simple way to think about it: Consider estimating only the mean. If your data are positively autocorrelated, and your first datum is, say, above the mean, then the second is more likely than not to be above the mean as well. After all, the point of being positively autocorrelated is that the next data point will be close to wherever the last data point was. The result of this is that your whole sample may be equally likely to be above or below the true mean, but the data will tend to cluster together more tightly than they would if they were independent. Hence, you SEs will be smaller than they would be with independent data, but your data will actually fluctuate (from sample to sample) more widely. That is, your SEs will be too small.

   Now consider that your data are negatively autocorrelated. Again, your first datum is above the mean. But because your data are negatively autocorrelated, your next datum will be that far below the mean. Thus, even your estimate of the mean from only your first two data will tend to be very close to the true mean. This process continues as you gather more data. Your data will scatter more widely than they would under independence, but counter-intuitively the amount that your sample mean will vary from the population mean will be less than under independence. That is, your SEs will be too large.