52
$\begingroup$

I suppose I get frustrated every time I hear someone say that non-normality of residuals and /or heteroskedasticity violates OLS assumptions. To estimate parameters in an OLS model neither of these assumptions are necessary by the Gauss-Markov theorem. I see how this matters in Hypothesis Testing for the OLS model, because assuming these things give us neat formulas for t-tests, F-tests, and more general Wald statistics.

But it is not too hard to do hypothesis testing without them. If we drop just homoskedasticity we can calculate robust standard errors and clustered standard errors easily. If we drop normality altogether we can use bootstrapping and, given another parametric specification for the error terms, likelihood ratio, and Lagrange multiplier tests.

It's just a shame that we teach it this way, because I see a lot of people struggling with assumptions they do not have to meet in the first place.

Why is it that we stress these assumptions so heavily when we have the ability to easily apply more robust techniques? Am I missing something important?

$\endgroup$
  • 2
    $\begingroup$ Seems to be a disciplinary thing. In my experience, at the extremes, Econometrics texts almost always cover what inferences each assumption buys and Psychology texts never seem to mention anything about the topic. $\endgroup$ – conjugateprior Dec 30 '14 at 23:35
  • 12
    $\begingroup$ Homoscedasticity is necessary for OLS to be BLUE though. $\endgroup$ – Momo Dec 30 '14 at 23:39
  • 4
    $\begingroup$ I think you are right, those assumptions receive undue attention. Failure of normality or conditional homoscedasticity. isn't close to as detrimental to most practitioners' inferential goals as endogeneity and poorly specified functional forms. $\endgroup$ – CloseToC Dec 30 '14 at 23:52
  • 2
    $\begingroup$ @CloseToC this is a very good point. Often we get so concerned about the statistical technicalities of the regression model that we forget the bigger picture...is my model correctly specified and exogenous? This should be emphasized repeatably as one of the primary questions to ask one's self when creating any model. $\endgroup$ – Zachary Blumenfeld Dec 31 '14 at 0:19
25
$\begingroup$

In Econometrics, we would say that non-normality violates the conditions of the Classical Normal Linear Regression Model, while heteroskedasticity violates both the assumptions of the CNLR and of the Classical Linear Regression Model.

But those that say "...violates OLS" are also justified: the name Ordinary Least-Squares comes from Gauss directly and essentially refers to normal errors. In other words "OLS" is not an acronym for least-squares estimation (which is a much more general principle and approach), but of the CNLR.

Ok, this was history, terminology and semantics. I understand the core of the OP's question as follows: "Why should we emphasize the ideal, if we have found solutions for the case when it is not present?" (Because the CNLR assumptions are ideal, in the sense that they provide excellent least-square estimator properties "off-the-shelf", and without the need to resort to asymptotic results. Remember also that OLS is maximum likelihood when the errors are normal).

As an ideal, it is a good place to start teaching. This is what we always do in teaching any kind of subject: "simple" situations are "ideal" situations, free of the complexities one will actually encounter in real life and real research, and for which no definite solutions exist.

And this is what I find problematic about the OP's post: he writes about robust standard errors and bootstrap as though they are "superior alternatives", or foolproof solutions to the lack of the said assumptions under discussion for which moreover the OP writes

"..assumptions that people do not have to meet"

Why? Because there are some methods of dealing with the situation, methods that have some validity of course, but they are far from ideal? Bootstrap and heteroskedasticity-robust standard errors are not the solutions -if they indeed were, they would have become the dominant paradigm, sending the CLR and the CNLR to the history books. But they are not.

So we start from the set of assumptions that guarantees those estimator properties that we have deemed important (it is another discussion whether the properties designated as desirable are indeed the ones that should be), so that we keep visible that any violation of them, has consequences which cannot be fully offset through the methods we have found in order to deal with the absence of these assumptions. It would be really dangerous, scientifically speaking, to convey the feeling that "we can bootstrap our way to the truth of the matter" -because, simply, we cannot.

So, they remain imperfect solutions to a problem, not an alternative and/or definitely superior way to do things. Therefore, we have first to teach the problem-free situation, then point to the possible problems, and then discuss possible solutions. Otherwise, we would elevate these solutions to a status they don't really have.

$\endgroup$
  • $\begingroup$ Hmmm, if that's what you had meant, you might try "fully proven". $\endgroup$ – gung Dec 31 '14 at 3:47
  • $\begingroup$ @gung No, no, the methods are "fully proven" in the mathematical sense, but not foolproof as regards what they actually deliver (this "little" detail about asymptotics again, and what their value is). Your correction was the right one. $\endgroup$ – Alecos Papadopoulos Dec 31 '14 at 3:54
22
$\begingroup$

If we had time in the class where we first introduce regression models to discuss bootstrapping and the other techniques that you mentioned (including all their assumptions, pitfalls, etc.), then I would agree with you that it is not necessary to talk about normality and homoscedasticity assumptions. But in truth, when regression is first introduced we do not have the time to talk about all those other things, so we would rather have the students be conservative and check for things that may not be needed and consult a statistician (or take another stats class or 2 or 3, ...) when the assumptions don't hold.

If you tell students that those assumptions don't matter except when ..., then most will only remember the don't matter part and not the important when parts.

If we have a case with unequal variances, then yes we can still fit a least squares line, but is it still the "best" line? or would it be better to consult someone with more experience/training on how to fit lines in that case. Even if we are happy with the least squares line, shouldn't we acknowledge that predictions will have different properties for different values of the predictor(s)? So checking for unequal variances is good for later interpretations, even if we don't need it for the tests/intervals/etc. that we are using.

$\endgroup$
  • $\begingroup$ I understand and appreciate what your saying especially that there is a significant time constraint. What I see at my institution, is when students struggle with these assumptions they often don't or can't receive necessary consulting. So they end up selecting projects on the basis of fitting model assumptions or inappropriately using the classical model to violating assumptions. I argue that by teaching more robust techniques, students would be less restricted in there choices and thus empowered to pursue projects they are actually passionate about. $\endgroup$ – Zachary Blumenfeld Dec 31 '14 at 0:00
  • 13
    $\begingroup$ You always start with an ideal case when teaching, then go into all kinds of complications. On PhD level economentrics they teach all kinds of weird stuff, but it takes time to get there. I don't think it's a problem of education that most people get off the train somewhere around MSc level. In actually, I would claim that the bigger issue is the pest infestation by half-baked "data scientists", with near zero knowledge of foundations of statistics applying fancy R packages left and right, having no clue of what they are doing and struggling to make a sense of the outputs. $\endgroup$ – Aksakal Dec 31 '14 at 2:28
  • $\begingroup$ @Aksakal where exactly do you see these large numbers of overconfident, underqualified analysts? Because what I encounter more often is nearly the opposite. People are afraid to try out the techniques they have learned unless first given approval from a perceived expert. Just on this site, I'm sure you've seen the many questions to the effect of "Am I allowed to..." or "Is it valid to...." when a more seasoned/constructive question would be "What would result if...." $\endgroup$ – rolando2 Jan 3 '15 at 20:44
18
$\begingroup$

1) rarely do people only want to estimate. Usually inference - CIs, PIs, tests - is the aim, or at least part of it (even if sometimes it's done relatively informally)

2) Things like the Gauss Markov theorem isn't necessarily much help -- if the distribution is sufficiently far from normal, a linear estimator is not much use. There's no point in getting the BLUE if no linear estimator is very good.

3) things like sandwich estimators involve a large number of implicit parameters. It may still be okay if you have a lot of data, but many times people don't.

4) Prediction intervals rely on the conditional distribution's shape including having a good handle on the variance at the observation - you can't quite so easily wave the details away with a PI.

5) things like bootstrapping are often handy for very large samples. They sometimes struggle in small samples -- and even in moderately sized samples, frequently we find that the actual coverage properties are nothing like advertized.

Which is to say -- few things are the sort of panacea people would like them to be. All of those things have their place, and there are certainly plenty of cases where (say) normality is not required, and where estimation and inference (tests and CIs) can reasonably be done without necessarily needing normality, constant variance and so on.

One thing that often seems to be forgotten is other parametric assumptions that could be made instead. Often people know enough about a situation to make a fairly decent parametric assumption (e.g. say... that the conditional response will tend to be right skew with s.d. pretty much proportional to mean might lead us to consider say a gamma or lognormal model); often this may deal with both the heteroskedasticity and the non-normality in one go.

A very useful tool is simulation -- with that we can examine the properties of our tools in situations very like those it appears our data may have arisen from, and so either use them in the comforting knowledge that they have good properties in those cases (or, sometimes, see that they don't work as well as we might hope).

$\endgroup$
  • $\begingroup$ "rarely do people only want to estimate" - in corporate finance and economics, people estimate a lot to tease out the impact of a certain variable. I've read a ton of papers, where authors don't even look at confidence intervals, they notice the significance, of course, under the normal assumptions. $\endgroup$ – Aksakal Dec 31 '14 at 4:33
  • 6
    $\begingroup$ @Aksakal well, yes, but if they are paying any attention to standard errors, or t-values, or p-values, etc ... then by my reckoning they don't only want to estimate. Whether they formally test/construct intervals or not, for that kind of looking at those other things -- however informally -- to be meaningful, they would have to have meaning in the first place. $\endgroup$ – Glen_b Dec 31 '14 at 4:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.