# Regression - How do I know if my residuals are normally distributed?

Performing a regression and need to find out if my residuals are normally distributed.

-
Take a look here as well stats.stackexchange.com/questions/58941/… –  MCP_infiltrator May 2 at 14:04

In practice you simply don't know (but they probably aren't). Not that non-normal residuals are necessarily a problem; it depends on how non-normal and how big your sample size is and how much you care about the impact on your inference.

You can see if the residuals are reasonably close to normal via a Q-Q plot.

A Q-Q plot isn't hard to generate in Excel.

If you take $r$ to be the ranks of the residuals (1 for smallest, 2 for second smallest, etc), then

$\Phi^{-1}(\frac{r-3/8}{n+1/4})$ is a good approximation for the expected normal order statistics. Plot the residuals against that transformation of their ranks, and it should look roughly like a straight line.

(where $\Phi^{-1}$ is the inverse cdf of a standard normal)

If you haven't used Q-Q plots before, I'd suggest generating a bunch of sets of random normal data (at several samples sizes) and seeing what the plots look like. (Roughly like points close to a straight line with some tendency to be a bit more noisy - wiggle a bit - at the ends)

Then generate skewed data, heavy tailed data, uniform data, bimodal data etc and see what the plots look like when data isn't normal. (Various kinds of curves and kinks, basically)

These plots are standard in most stats packages.

Here's one done in R:

Here's one I just generated in Excel via the above method:

(not the same set of data both times)

You can see the points form a straightish line ... that's because the data was actually normal.

Here's one that's not normal (it's quite right skew):

If you ever happen to be using something that has neither Q-Q plots nor inverse normal cdf functions, proceed as above up to the ranking stage, then find $p=\frac{r-3/8}{n+1/4}$ but use the Tukey lambda approximation to the inverse normal cdf.

Actually, there are two such that have been in popular use:

$\Phi^{-1}(p) \approx 5.05 (p^{0.135} - (1-p)^{0.135})$

$\Phi^{-1}(p) \approx 4.91 (p^{0.14} - (1-p)^{0.14})$

(Either is quite adequate, but my recollection is that the second seemed to work slightly better. I believe Tukey used 1/0.1975 = 5.063 in the first one instead of 5.05)

-

Using plots of simulated data to get an impression of how to interpret a Q-Q plot as propsed by @Glen_b is an excelent idea.

You can also use such simulated curves as a background in your final graph. That way it is easier to compare the diviations from the diagonal in your observed residuals with the kind of variation from the diagonal line one could expect when the residuals were draws from a real normal distribution. See for example the graph below:

The details:

I made this graph in Stata. For convenience I used for the plotting position $p=\frac{r-.5}{n}$, the default for qplot. For those who have Stata and whish to play with it, here is the code (it requires the user written components qplot and the lean1 scheme, both can be found using findit):

// make sure the random draws can be replicated
set seed 12345

sysuse auto, clear

// do a regression
reg price mpg foreign i.rep78

// predict the residuals
predict resid, resid

// create 19 random draws
forvalues i = 1/19 {
gen residi' = rnormal(0,e(rmse))
}

//create the Q-Q plot
qplot resid? resid?? resid, trscale(invnorm(@)*e(rmse)) ///
lcolor( : display _dup(19) "gs12 "' black)         ///
msymbol(: display _dup(19) "none "' oh   )         ///
connect(: display _dup(19) "l "'    .    )         ///
lpattern(solid...)                                  ///
legend(order(20 "observed" 19 "simulated" )         ///
subtitle(residuals))                         ///
aspect(1) scheme(lean1)

-
That's a very nice plot. –  Glen_b May 6 at 22:33
The point is that if you have enough observations (say > 100) normality/Gaussianity just isn't an issue. A second point is that there are many normality tests, and the JB test is about the worst, even worse then the Kolmogorov-Smirnov test. With bad I mean that the $p$-values don't mean what they should mean in a statistical test. for a simulation showing this, see here. One normality test that performs fairly well is the Doornik-Hansen, but there are others. –  Maarten Buis May 2 at 15:45