Why do we care so much about normally distributed error terms (and homoskedasticity) in linear regression when we don't have to?

Question

I suppose I get frustrated every time I hear someone say that non-normality of residuals and /or heteroskedasticity violates OLS assumptions. To estimate parameters in an OLS model neither of these assumptions are necessary by the Gauss-Markov theorem. I see how this matters in Hypothesis Testing for the OLS model, because assuming these things give us neat formulas for t-tests, F-tests, and more general Wald statistics.

But it is not too hard to do hypothesis testing without them. If we drop just homoskedasticity we can calculate robust standard errors and clustered standard errors easily. If we drop normality altogether we can use bootstrapping and, given another parametric specification for the error terms, likelihood ratio, and Lagrange multiplier tests.

It's just a shame that we teach it this way, because I see a lot of people struggling with assumptions they do not have to meet in the first place.

Why is it that we stress these assumptions so heavily when we have the ability to easily apply more robust techniques? Am I missing something important?

Answer 1

In Econometrics, we would say that non-normality violates the conditions of the Classical Normal Linear Regression Model, while heteroskedasticity violates both the assumptions of the CNLR and of the Classical Linear Regression Model.

But those that say "...violates OLS" are also justified: the name Ordinary Least-Squares comes from Gauss directly and essentially refers to normal errors. In other words "OLS" is not an acronym for least-squares estimation (which is a much more general principle and approach), but of the CNLR.

Ok, this was history, terminology and semantics. I understand the core of the OP's question as follows: "Why should we emphasize the ideal, if we have found solutions for the case when it is not present?" (Because the CNLR assumptions are ideal, in the sense that they provide excellent least-square estimator properties "off-the-shelf", and without the need to resort to asymptotic results. Remember also that OLS is maximum likelihood when the errors are normal).

As an ideal, it is a good place to start teaching. This is what we always do in teaching any kind of subject: "simple" situations are "ideal" situations, free of the complexities one will actually encounter in real life and real research, and for which no definite solutions exist.

And this is what I find problematic about the OP's post: he writes about robust standard errors and bootstrap as though they are "superior alternatives", or foolproof solutions to the lack of the said assumptions under discussion for which moreover the OP writes

"..assumptions that people do not have to meet"

Why? Because there are some methods of dealing with the situation, methods that have some validity of course, but they are far from ideal? Bootstrap and heteroskedasticity-robust standard errors are not the solutions -if they indeed were, they would have become the dominant paradigm, sending the CLR and the CNLR to the history books. But they are not.

So we start from the set of assumptions that guarantees those estimator properties that we have deemed important (it is another discussion whether the properties designated as desirable are indeed the ones that should be), so that we keep visible that any violation of them, has consequences which cannot be fully offset through the methods we have found in order to deal with the absence of these assumptions. It would be really dangerous, scientifically speaking, to convey the feeling that "we can bootstrap our way to the truth of the matter" -because, simply, we cannot.

So, they remain imperfect solutions to a problem, not an alternative and/or definitely superior way to do things. Therefore, we have first to teach the problem-free situation, then point to the possible problems, and then discuss possible solutions. Otherwise, we would elevate these solutions to a status they don't really have.

Answer 2

If we had time in the class where we first introduce regression models to discuss bootstrapping and the other techniques that you mentioned (including all their assumptions, pitfalls, etc.), then I would agree with you that it is not necessary to talk about normality and homoscedasticity assumptions. But in truth, when regression is first introduced we do not have the time to talk about all those other things, so we would rather have the students be conservative and check for things that may not be needed and consult a statistician (or take another stats class or 2 or 3, ...) when the assumptions don't hold.

If you tell students that those assumptions don't matter except when ..., then most will only remember the don't matter part and not the important when parts.

If we have a case with unequal variances, then yes we can still fit a least squares line, but is it still the "best" line? or would it be better to consult someone with more experience/training on how to fit lines in that case. Even if we are happy with the least squares line, shouldn't we acknowledge that predictions will have different properties for different values of the predictor(s)? So checking for unequal variances is good for later interpretations, even if we don't need it for the tests/intervals/etc. that we are using.

Answer 3

1) rarely do people only want to estimate. Usually inference - CIs, PIs, tests - is the aim, or at least part of it (even if sometimes it's done relatively informally)

2) Things like the Gauss Markov theorem isn't necessarily much help -- if the distribution is sufficiently far from normal, a linear estimator is not much use. There's no point in getting the BLUE if no linear estimator is very good.

3) things like sandwich estimators involve a large number of implicit parameters. It may still be okay if you have a lot of data, but many times people don't.

4) Prediction intervals rely on the conditional distribution's shape including having a good handle on the variance at the observation - you can't quite so easily wave the details away with a PI.

5) things like bootstrapping are often handy for very large samples. They sometimes struggle in small samples -- and even in moderately sized samples, frequently we find that the actual coverage properties are nothing like advertized.

Which is to say -- few things are the sort of panacea people would like them to be. All of those things have their place, and there are certainly plenty of cases where (say) normality is not required, and where estimation and inference (tests and CIs) can reasonably be done without necessarily needing normality, constant variance and so on.

One thing that often seems to be forgotten is other parametric assumptions that could be made instead. Often people know enough about a situation to make a fairly decent parametric assumption (e.g. say... that the conditional response will tend to be right skew with s.d. pretty much proportional to mean might lead us to consider say a gamma or lognormal model); often this may deal with both the heteroskedasticity and the non-normality in one go.

A very useful tool is simulation -- with that we can examine the properties of our tools in situations very like those it appears our data may have arisen from, and so either use them in the comforting knowledge that they have good properties in those cases (or, sometimes, see that they don't work as well as we might hope).