0
$\begingroup$

I just started learning about simple linear regression, and I have a question about one of its assumptions.

One of the assumptions is that the errors are normally distributed. Does this mean that if I get every $y-\hat{y}$ point, those points should be distributed as a mound shape?

Improve this question
$\endgroup$
  • $\begingroup$ The normality assumption only means that the MLE is the least squares solution. Without normality the least squares estimate can still be BLUE (best linear unbiased estimate). $\endgroup$ – Michael R. Chernick Nov 24 '17 at 23:54
  • $\begingroup$ It means that if you fit all of the signal and none of the noise (i.e. have a perfect model) then $y -/hat{y}$ should be distributed as gaussian. $\endgroup$ – David Kozak Nov 25 '17 at 0:52
  • $\begingroup$ If 'mound shape' = bell curve, then yes. $\endgroup$ – eSurfsnake Nov 25 '17 at 2:25
  • $\begingroup$ @eSurfsnake, actually, no. Consider $\hat{y}=0 \forall x$. It's a badly misspecified model but it is one. And it does not satisfy $\hat{\epsilon} \sim N(0,\sigma^2)$ in general. $\endgroup$ – David Kozak Nov 25 '17 at 3:19
  • $\begingroup$ @eSurfsnake, In answering questions like this, it is essential that you distinguish "errors" (which are an additive random variable in the model) from the residuals, which is what this question appears to be concerned with. Errors don't depend on the fit, but residuals do. $\endgroup$ – whuber♦ Nov 25 '17 at 18:26
2
$\begingroup$

The assumption* relates to the errors rather than the residuals, but if the assumption is satisfied, you would expect the residuals to look close to normal.

While widely used by people who use a few particular pieces of software, histograms are a very blunt diagnostic tool for assessing normality; I tend to use Q-Q plots for that purpose while keeping in mnd that no model is perfect (its more about how much impact the non normality might have)

* see my comment in relation to when you use that assumption.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.