What are the dangers of violating the homoscedasticity assumption for linear regression?

Question

As an example, consider the ChickWeight data set in R. The variance obviously grows over time, so if I use a simple linear regression like:

m <- lm(weight ~ Time*Diet, data=ChickWeight)

My questions:

1. Which aspects of the model will be questionable?
2. Are the problems limited to extrapolating outside the Time range?
3. How tolerant is linear regression to violation of this assumption (i.e., how heteroscedastic does it have to be to cause problems)?

Answer 1

The linear model (or "ordinary least squares") still has its unbiasedness property in this case.

In the face of heteroskedasticity in error terms, you still have unbiased parameter estimates but you get a biased estimate of the covariance matrix: your inference (i.e. parameter tests and confidence intervals) could be incorrect. The common fix is to use a robust method for computing the covariance matrix aka standard errors. Which one you use is somewhat domain-dependent but White's method is a start.

And for completeness, serial correlation of error terms is worse as it will lead to biased parameter estimates.

Answer 2

Homoscedasticity is one of the Gauss Markov assumptions that are required for OLS to be the best linear unbiased estimator (BLUE).

The Gauss-Markov Theorem is telling us that the least squares estimator for the coefficients $\beta$ is unbiased and has minimum variance among all unbiased linear estimators, given that we fulfill all Gauss-Markov assumptions. You can find more information on the Gauss-Markov Theorem including the mathematical proof of the theorem here. Additionally, you can find a complete list of the OLS assumptions including explanations what happens in case they are violated here.

Briefly summarizing the information from the websites above, heteroscedasticity does not introduce a bias in the estimates of your coefficients. However, given heteroscedasticity, you are not able to properly estimate the variance-covariance matrix. Hence, the standard errors of the coefficients are wrong. This means that one cannot compute any t-statistics and p-values and consequently hypothesis testing is not possible. Overall, under heteroscedasticity OLS loses its efficiency and is not BLUE anymore.

However, heteroscedasticity is not the end of the world. Fortunately, correcting for heteroscedasticity is not difficult. The sandwich estimator allows you to estimate consistent standard errors for the coefficients. Nevertheless, computing the standard errors via the sandwich estimator comes at a cost. The estimator is not very efficient and standard errors might be very large. One way to gain back some of the efficiency is to cluster standard errors if possible.

You can find more detailed information on this subject on the websites I referred above.

Answer 3

It is good to remember that having unbiased estimators does not mean that the model is "right". In many situations, the least squares criterion for regression coefficient estimation gives rise to a model that either has (1) regression coefficients that don't have the right meaning or (2) predictions that are tilted towards minimizing large errors but that make up for it by having many small errors. For example, some analysts believe that even when transforming to $\log(Y)$ makes the model fit well it is valid to predict $Y$ using OLS because estimates are unbiased. This will minimize the sum of squared errors but partition the effects across the $\beta$s incorrectly and result in a non-competitive sum of absolute errors. Sometimes lack of constancy of variance signals a more fundamental modeling problem.

When looking at competing models (e.g., for $Y$ vs. $\log(Y)$ vs. ordinal regression) I like to compare predictive accuracy using measures that were not optimized by definition by the fitting process.

Answer 4

Absence of homoscedasticity may give unreliable standard error estimates of the parameters. Parameter estimates are unbiased. But the estimates may not efficient(not BLUE). You can find some more in the following link

Answer 5

There is good information here in the other answers, particularly to your first question. I thought I would add some complimentary information regarding your last two questions.

1. The problems associated with heteroscedasticity are not limited to extrapolation. Since they primarily involve confidence intervals, p-values, and prediction limits being incorrect, they apply throughout the range of your data.
2. Strictly speaking, the problems associated with heteroscedasticity exist with even the smallest amount of heteroscedasticity. However, as you might suspect, with very little heteroscedasticity, the problems are very small as well. There is no true 'bright line' where heteroscedasticity becomes too much, but a rule of thumb is that linear models are not too affected by heteroscedasticity when the largest variance is $\le 4\times$ the smallest variance.