Normality within cells and non normality of residuals

Question

In testing for the normality assumption for 2 way ANOVA, the question whether normality within cells implies normality of residuals has been asked before and I have found so far different answers from different forums and still it's not clear to me.

I have the data of 6 patients that have been treated twice with 2 different drugs and each with 3 different levels of dose. So there are 2 drugs with 3 levels and I end up with six cells.

Doing 2 way anova I get a significant interaction and also the data passes the homoscedasticity test (Levene test). The normality assumption is true when is tested within each cell (6 cells were tested separately). However, when I do the test over the residuals of the anova model the result is the opposite (p<0.01).

Can someone please explain which of these two normality tests should I rely on (over residuals or within each cell) and if the normality of residuals matters.

How can I confirm that I am allowed to run 2 way ANOVA? Does this difference stem from the low sample size of my population?

Answer 1

Well, that depends on what you want to do with the results.

From this: "...in ANOVA, you actually have two options for testing normality. If there really are many values of Y for each value of X (each group), and there really are only a few groups (say, four or fewer), go ahead and check normality separately for each group.

But if you have many groups (a 2x2x3 ANOVA has 12 groups) or if there are few observations per group (it’s hard to check normality on only 20 data points), it’s often easier to just use the residuals and check them all together."

Which means that 1) If you have less than 20 in each cell, the power for checking normality in each cell is too low to be strongly indicative of residual structure. 2) You can do ANOVA but the results may be wonky. Some suggest not testing for normality because mean values from ANOVA of data are not very sensitive to non-normality. Non normal extreme values, "outliers", and skewness are often more important than non-normality itself.

An alternative may be to rank the entire data, but that depends on the data all having the same scale. So, what does the data look like? Ranking will generally normalize the residuals, if appropriate, and make a non-parametric ANOVA possible. There are also a number of non-parametric alternatives to ANOVA. When in doubt, do a non-parametric test as well as parametric ANOVA and compare them.

BTW, there are more flavors of ANOVA than of Baskin-Robins ice cream, so it depends somewhat on what information you are attempting to extract. For a brief introduction (SPSS) of ANOVA assumptions see this link.

Answer 2

Welcome to CV.

Certainly @Carl's answer raises good points. One clear problem with all tests of normality is that the p value depends partly on sample size while the assumption of normality does not. That is, the normality assumption is not that the deviation from normality is not significant, it is that the errors are normal.

In the old days, before fast computers, it made sense to try to assess normality because the methods that don't assume normality were hard to do. Now, they are not.

So, I suggest doing (say) robust regression or quantile regression and comparing the results with your ANOVA results. If they are similar, all is well. If they are quite different, then you have to think.