7
$\begingroup$

There are a lot of similar questions here but I have not found an answer to this specific question.

Source: for example in https://peopleanalytics-regression-book.org/linear-reg-ols.html#norm-dist-assum the author (a mathematician) says:

In an appropriate model we expect our errors to be random, so we would therefore expect our residuals to be normally distributed over sufficient numbers of observations.

The author then goes on to apply qqnorm(newmodel$residuals) to the data for diagnostics.

If you plot a model in R (plot(mymodel)), you get a bunch of diagnostic plots, the second of which is standardized residuals plotted against the theoretical quantiles - so essentially the same.

But why? What is the reasoning behind the residuals being normally distributed, and not just randomly, without having a recognised distribution at all, or some other distribution? Stats textbooks treat this as if it was obvious - could someone explain, please?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ The author made the same common mistake about the central limit theorem that I discuss here. $\endgroup$
    – Dave
    Jun 5, 2022 at 15:17
  • $\begingroup$ @Dave - that's a great thread (+1). $\endgroup$
    – jbowman
    Jun 5, 2022 at 15:32
  • $\begingroup$ Thanks to all who took the time/trouble to respond. @Dave, I will look at the thread in a little while. Just to emphasize that since one of the R built-in diagnostic plots is also checking the residuals against theoretical quantiles, I thought there was some reasoning behind it...? $\endgroup$
    – Reader 123
    Jun 5, 2022 at 15:42
  • 2
    $\begingroup$ Nope, just mistaken reasoning. The reason for checking the residuals etc. Is that if the residuals are far enough away from Normality, maybe you should try a different estimation procedure that might be better (like a robust one), or check outlying data points to make sure you understand them, for example. Maybe your model is incomplete and that shows up as a patch of outliers. Lots of possibilities. $\endgroup$
    – jbowman
    Jun 5, 2022 at 17:49
  • $\begingroup$ I once answered a why-Normal-errors question, where many other answers focused on why empirical variables are Normal. I'm not sure the points I summarized there explain why residuals are Normal, so I'm following this question in the hopes someone provides a basis specifically for expecting that third idea. $\endgroup$
    – J.G.
    Jun 6, 2022 at 6:25

1 Answer 1

Reset to default
15
$\begingroup$

That author is writing nonsense. Just because errors are random doesn't mean that if you have a lot of them they will be normally distributed. It is absolutely not the case that OLS requires normally distributed residuals; its objective is "Least Squares", and minimizing the sum of squared deviations from the estimated values in no way requires any particular distribution for the residuals. See, for example, Regression when the OLS residuals are not normally distributed , and some of the associated questions linked to in comments.

It is true that if the underlying errors follow a Normal distribution, and are independent and identically distributed, then the OLS estimator is also the maximum likelihood estimator. But that in no way justifies the author's statement quoted in the original post.

Edit: For more, read @Glen_b's comment below.

Improve this answer
$\endgroup$
4
  • 10
    $\begingroup$ +1 ... One might also remark that the "usual" tests and intervals are derived under the assumption of normality but tests and intervals under other distributional assumptions are possible (as indeed are some nonparametric approaches), without necessarily requiring that we move away from least squares (for all that there may be some loss of efficiency under different parametric assumptions) $\endgroup$
    – Glen_b
    Jun 5, 2022 at 23:38
  • 1
    $\begingroup$ @Glen_b, this holds for small samples, while in large samples the central limit theorem allows us to drop the assumption of normality of errors in many relevant tests. $\endgroup$
    – Richard Hardy
    Jun 6, 2022 at 10:12
  • $\begingroup$ Keep in mind the exposition is made under assumptions of a specific parametric tests (see the sections above) $\endgroup$
    – Firebug
    Jun 6, 2022 at 12:23
  • 1
    $\begingroup$ Thank you very much to all who took the trouble to comment and contribute. $\endgroup$
    – Reader 123
    Jun 7, 2022 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.