# What do normal residuals mean and what does this tell me about my data?

Pretty basic question:

What does a normal distribution of residuals from a linear regression mean? In terms of, how does this reflect on my original data from the regression?

I'm totally stumped, thanks guys

Linear regression in fact models the conditional expected values of your outcome. That means: if you knew the true values of the regression parameters (say $\beta_0$ and $\beta_1$), given a value of your predictor X, filling that out in the equation $$E[Y|X] = \beta_0 + \beta_1 X$$ will have you calculate the expected value for $Y$ over all (possible) observations that have this given value for $X$.

However: you don't really expect any single $Y$ value for that given $X$ value to be exactly equal to the (conditional) mean. Not because your model is wrong, but because there are some effects you have not accounted for (e.g. measuring error). So these $Y$ values for a given $X$ values will fluctuate around the mean value (i.e. geometrically: around the point of the regression line for that $X$).

The normality assumption, now, says that the difference between the $Y$s and their matching $E[Y|X]$ follows a normal distribution with mean zero. This means, if you have an $X$ value, then you can sample a $Y$ value by first calculating $\beta_0 + \beta_1 X$ (i.e. again $E[Y|X]$, the point on the regression line), next sampling $\epsilon$ from that normal distribution and adding them: $$Y'=E[Y|X] + \epsilon$$

In short: this normal distribution represents the variability in your outcome on top of the variability explained by the model.

Note: in most datasets, you don't have multiple $Y$ values for any given $X$ (unless your predictor set is categorical), but this normality goes for the whole population, not just the observations in your dataset.

Note: I've done the reasoning for linear regression with one predictor, but the same goes for more: just replace "line" with "hyperplane" in the above.

• This is a great explanation! One question though: e being normally distributed would mean that you assume that the most likely values for e are between -1 and +1 (after they were standardized)? So you basically use a normal distribution instead of, say, a poisson distribution, because the normal distribution better models how these values behave in real life? Jun 22, 2018 at 20:55
• I'm aware that linear regression can be equivalently derived in multiple ways, one of which is this MLE estimate type motivation. But how would you arrive at a similar reason for normality of residuals, starting from the motivation of trying to minimize sum of squared errors? Mar 13, 2021 at 19:16
• This confusingly implies the errors $\epsilon$ are the residuals in a regression--but they are not. This is an issue because many people fail to distinguish the concepts.
– whuber
Jul 13, 2022 at 13:01
• @whuber: So then the residuals are then the errors in the linear approximation obtained from the sample data, not from the " actual" regression line which is unknowable?
– MSIS
Nov 27, 2023 at 21:22
• @MSIS That sounds right.
– whuber
Nov 27, 2023 at 22:15

Normality of the residuals is an assumption of running a linear model. So, if your residuals are normal, it means that your assumption is valid and model inference (confidence intervals, model predictions) should also be valid. It's that simple!

• The normality assumption is about unobservable error (hence the need for an assumption), not about observable residuals. Mar 28, 2013 at 15:37
• Yeah, but you use the residuals to test your assumption about the unobservable error. Mar 29, 2013 at 12:38
• I do not agree that normal residuals guarantee a valid regression model. Suppose you have a circular Gaussian model with X and Y error that are equal. Then the regression line confidence interval is $-\infty \text { to } \infty$. That is hardly the only counter example, there are many more.
– Carl
Aug 6, 2016 at 0:00