Regression - How do I know if my residuals are normally distributed?

Question

Performing a regression and need to find out if my residuals are normally distributed.

Answer 1

In practice you simply don't know (but they probably aren't). Not that non-normal residuals are necessarily a problem; it depends on how non-normal and how big your sample size is and how much you care about the impact on your inference.

You can see if the residuals are reasonably close to normal via a Q-Q plot.

A Q-Q plot isn't hard to generate in Excel.

If you take $r$ to be the ranks of the residuals (1 for smallest, 2 for second smallest, etc), then

$\Phi^{-1}(\frac{r-3/8}{n+1/4})$ is a good approximation for the expected normal order statistics. Plot the residuals against that transformation of their ranks, and it should look roughly like a straight line.

(where $\Phi^{-1}$ is the inverse cdf of a standard normal)

If you haven't used Q-Q plots before, I'd suggest generating a bunch of sets of random normal data (at several samples sizes) and seeing what the plots look like. (Roughly like points close to a straight line with some tendency to be a bit more noisy - wiggle a bit - at the ends)

Then generate skewed data, heavy tailed data, uniform data, bimodal data etc and see what the plots look like when data isn't normal. (Various kinds of curves and kinks, basically)

These plots are standard in most stats packages.

Here's one done in R:

Here's one I just generated in Excel via the above method:

(not the same set of data both times)

You can see the points form a straightish line ... that's because the data was actually normal.

Here's one that's not normal (it's quite right skew):

If you ever happen to be using something that has neither Q-Q plots nor inverse normal cdf functions, proceed as above up to the ranking stage, then find $p=\frac{r-3/8}{n+1/4}$ but use the Tukey lambda approximation to the inverse normal cdf.

Actually, there are two such that have been in popular use:

$\Phi^{-1}(p) \approx 5.05 (p^{0.135} - (1-p)^{0.135})$

$\Phi^{-1}(p) \approx 4.91 (p^{0.14} - (1-p)^{0.14})$

(Either is quite adequate, but my recollection is that the second seemed to work slightly better. I believe Tukey used 1/0.1975 = 5.063 in the first one instead of 5.05)

Answer 2

Using plots of simulated data to get an impression of how to interpret a Q-Q plot as propsed by @Glen_b is an excelent idea.

You can also use such simulated curves as a background in your final graph. That way it is easier to compare the diviations from the diagonal in your observed residuals with the kind of variation from the diagonal line one could expect when the residuals were draws from a real normal distribution. See for example the graph below:

---

The details:

I made this graph in Stata. For convenience I used for the plotting position $p=\frac{r-.5}{n}$, the default for qplot. For those who have Stata and whish to play with it, here is the code (it requires the user written components qplot and the lean1 scheme, both can be found using findit):

// make sure the random draws can be replicated
set seed 12345

// load some data
sysuse auto, clear

// do a regression
reg price mpg foreign i.rep78

// predict the residuals
predict resid, resid

// create 19 random draws
forvalues i = 1/19 {
    gen resid`i' = rnormal(0,e(rmse))
}

//create the Q-Q plot
qplot resid? resid?? resid, trscale(invnorm(@)*e(rmse)) ///
    lcolor( `: display _dup(19) "gs12 "' black)         ///
    msymbol(`: display _dup(19) "none "' oh   )         ///
    connect(`: display _dup(19) "l "'    .    )         ///
    lpattern(solid...)                                  ///
    legend(order(20 "observed" 19 "simulated" )         ///
           subtitle(residuals))                         ///
    aspect(1) scheme(lean1)

Answer 3

There are normality tests, such as Jarque-Bera test, you can find them in any statistical package.

There was a bunch of comments recommending you not to test for normality. I agree that you don't have to test for normality if you don't use this assumption. However, if you use normality assumption then you better demonstrate that it holds. In some regulated industries you must produce the evidence that your assumptions, such as normality, holds.