ANOVA assumption normality/normal distribution of residuals

• Independence of cases – this is an assumption of the model that simplifies the statistical analysis.
• Normality – the distributions of the residuals are normal.
• Equality (or "homogeneity") of variances, called homoscedasticity...

Point of interest here is the second assumption. Several sources list the assumption differently. Some say normality of the raw data, some claim of residuals.

Several questions pop up:

• are normality and normal distribution of residuals the same person (based on Wikipedia entry, I would claim normality is a property, and does not pertain residuals directly (but can be a property of residuals (deeply nested text within brackets, freaky)))?
• if not, which assumption should hold? One? Both?
• if the assumption of normally distributed residuals is the right one, are we making a grave mistake by checking only the histogram of raw values for normality?
-
You can pretty much ignore anything else those sources that say if they claim the raw data needs to be normally distributed. And who said "we" were only checking the raw values with histograms, anyway. Are you in one of those Six Sigma classes??? –  DWin Jan 18 '11 at 20:58
@Andy W: I've just added a link to what appears to be the relevant section of the Wikipedia article on ANOVA. –  onestop Jan 18 '11 at 20:58
@DWin: blog.markanthonylawson.com/?p=296 (sorry, completely off-topic but couldn't resist) –  onestop Jan 18 '11 at 21:02
@onestop thank you. I only requested the link because I am lazy and did not want to look up ANOVA on wikipedia myself, not because it is essential for the question. –  Andy W Jan 19 '11 at 3:44
Related question here: what-if-residuals-are-normally-distributed-but-y-is-not. –  gung Sep 13 '12 at 1:38

Let's assume this is a fixed effects model. (The advice doesn't really change for random-effects models, it just gets a little more complicated.)

1. No, normality and normal distribution of residual are not the same. Suppose you measured yield from a crop with and without a fertilizer application. In plots without fertilizer the yield ranged from 70 to 130. In two plots with fertilizer the yield ranged from 470 to 530. The distribution of results is strongly non-normal: it's clustered at two locations related to the fertilizer application. Suppose further the average yields are 100 and 500, respectively. Then all residuals range from -30 to +30. They might (or might not) be normally distributed, but obviously this is a completely different distribution.

2. The distribution of the residuals matters, because they reflect the random part of the model. Note also that the p-values are computed from F (or t) statistics and those depend on residuals, not on the original values.

3. If there are significant and important effects in the data (as in this example), then you might be making a "grave" mistake. You could, by luck, make the correct determination: that is, by looking at the raw data you will seing a mixture of distributions and this can look normal (or not). The point is that what you're looking it is not relevant.

ANOVA residuals don't have to be anywhere close to normal in order to fit the model. However, near-normality of the residuals is essential for p-values computed from the F-distribution to be meaningful.

-
I think there is important points to add: in an ANOVA, the normality within each group (not overall) is equivalent to the normality of the residuals. –  Aniko Jan 18 '11 at 20:00
@Aniko Could you please elaborate on what you mean by "equivalent" in your comment? It is almost tautological that normality within a group is the same as normality of that group's residuals, but it is false that normality separately within each group implies (or is implied by) normality of the residuals. –  whuber Jan 18 '11 at 21:39
I really meant the tautological sense: if the groups are normal then the residuals are normal. The reverse is only true if homoscedascity is added (as in ANOVA). I don't mean to advocate for checking the groups instead of the residuals, but I think this is the underlying reason for the varying phrasing of the assumptions. –  Aniko Jan 18 '11 at 22:14
@Aniko Thank you for the clarification. –  whuber Jan 18 '11 at 23:27

Standard Classical one-way ANOVA can be viewed as an extension to the classical "2-sample T-test" to an "n-sample T-test". This can be seen from comparing a one-way ANOVA with only two groups to the classical 2-sample T-test.

I think where you are getting confused is that (under the assumptions of the model) the residuals and the raw data are BOTH normally distributed. However the raw data consist of normal distributions with different means (unless all the effects are exactly the same) but the same variance. The residuals on the other hand have the same normal distribution. This comes from the third assumption of homoscedasticity.

This is because the normal distribution is decomposable into a mean and variance components. If $Y_{ij}$ has a normal distribution with mean $\mu_{j}$ and variance $\sigma^2$ can be written as $Y_{ij}=\mu_{j}+\sigma\epsilon_{ij}$ where $\epsilon_{ij}$ has a standard normal distribution.

While ANOVA is derivable from the assumption of normality, I think (but am unsure) it can be replaced by an assumption of linearity (along the Best Linear Unbiased Estimator (BLUE) lines of estimation, where "BEST" is interpreted as minimum mean square error). I believe this basically involves replacing the distribution for $\epsilon_{ij}$ with any mutually independent distribution (over all i and j) which has mean 0 and variance 1.

In terms of looking at your raw data, it should look normal when plotted separately for each factor level in your model. This means plotting $Y_{ij}$ for each j on a separate graph.

-
+1 for pointing out (in the last paragraph) the assumption of homoscedasticity. –  whuber Jan 19 '11 at 3:55
Does it mean that if we have let say n dependent groups to compare we need to check their residuals separately (resulting in n groups of residuals)? –  stan Dec 10 '12 at 17:43

In the one-way case with $p$ groups of size $n_{j}$: $F = \frac{SS_{b} / df_{b}}{SS_{w} / df_{w}}$ where

$SS_{b} = \sum_{j=1}^{p}{n_{j} (M - M_{j}})^{2}$ and

$SS_{w} = \sum_{j=1}^{p}\sum_{i=1}^{n_{j}}{(y_{ij} - M_{j})^{2}}$

$F$ follows an $F$-distribution if $SS_{b} / df_{b}$ and $SS_{w} / df_{w}$ are independent, $\chi^{2}$-distributed variables with $df_{b}$ and $df_{w}$ degrees of freedom, respectively. This is the case when $SS_{b}$ and $SS_{w}$ are the sum of squared independent normal variables with mean $0$ and equal scale. Thus $M-M_{j}$ and $y_{ij}-M_{j}$ must be normally distributed.

$y_{i(j)} - M_{j}$ is the residual from the full model ($Y = \mu_{j} + \epsilon = \mu + \alpha_{j} + \epsilon$), $y_{i(j)} - M$ is the residual from the restricted model ($Y = \mu + \epsilon$). The difference of these residuals is $M - M_{j}$.

EDIT to reflect clarification by @onestop: under $H_{0}$ all true group means are equal (and thus equal to $M$), thus normality of the group-level residuals $y_{i(j)} - M_{j}$ implies normality of $M - M_{j}$ as well. The DV values themselves need not be normally distributed.

-
The assumption is that these $SS$ are $\chi^2$-distributed under the null hypothesis, which is that the group means are all equal, i.e. $M_j=M$ for all $j$. When this is the case, $y_{ij}-M_j$ is normal implies $M_j-M$ is normal. So you only need to check the first, i.e. that the observation-level residuals are normal. –  onestop Jan 18 '11 at 20:51
@onestop Edited to reflect your clarification, thanks! –  caracal Jan 19 '11 at 9:34