I am currently trying to perform a hypothesis test on the difference between four means. Initially I was trying to use ANOVA but then realised I may not meet the assumptions for this test and may need to use the non-parametric version Kruskal-Wallis. I started trying to test the ANOVA assumptions and found that each group had the same variance but using the Shapiro Test I found that my groups were not normally distributed (my sample sizes are small so as far as I'm aware the fact that the test could detect differences in normality suggests there are significant differences from the Normal distribution). I originally concluded that this meant I should use Kruskal-Wallis however looking back at the assumptions for the ANOVA test I realised it actually says to check the normality of the residuals. I'm a little confused by this as I thought not having the assumption of being normally distributed automatically meant we needed to use a non-parametric test. Can someone please explain to me if I do also need to check the normality of residuals and if I do why I need to and how would I check this?
2 Answers
It is important to check for residuals rather than normality of the collection of all responses.
Mixture of normal observations need not be normal. I will give an illustration with $g = 4$ groups and $n = 10$ replications in each group. Data are simulated as normal with several different means and equal variances, but a Shapiro-Wilk test rejects normality for the $gn = 40$ observations taken together.
set.seed(1234)
g = 4; n = 10
x1 = rnorm(10, 20, 5); x2 = rnorm(10, 25, 5)
x3 = rnorm(10, 35, 5); x4 = rnorm(10, 50, 5)
x = c(x1, x2, x3, x4)
shapiro.test(x)
Shapiro-Wilk normality test
data: x
W = 0.93777, p-value = 0.0291
Taken together the 40 observations have a normal mixture distribution, which need not be normal. Perhaps Wikipedia on mixture distributions, especially the figure near the top of the page.
Looking at residuals. However, for this simple model, the residuals are found by subtracting the mean for each group from each observation in the group. The 40 residuals pass the Shapiro-Wilk test.
r1 = x1 - mean(x1); r2 = x2 - mean(x2)
r3 = x3 - mean(x3); r4 = x4 - mean(x4)
r = c(r1, r2, r3, r4)
shapiro.test(r)
Shapiro-Wilk normality test
data: r
W = 0.98231, p-value = 0.7743
ANOVA Significant. Because the group population means are quite different, a one-way ANOVA on my fake data shows a highly significant effect.
gp = as.factor(rep(1:4, each=10))
lm.out = lm(x ~ gp); anova(lm.out)
Analysis of Variance Table
Response: x
Df Sum Sq Mean Sq F value Pr(>F)
gp 3 5655.9 1885.31 62.167 2.596e-14 ***
Residuals 36 1091.8 30.33
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
It is precisely in the cases where there is a significant effect that the aggregated data from all groups are likely to fail the Shapiro-Wilk normality test.
-
$\begingroup$ +1, but I want to clarify that this depends on the "groups" having the same variance. Also, it is unclear, but I think that the OP tested normality of each group separately. If so, this doesn't fully address the question. $\endgroup$– knrumseyMar 1, 2019 at 18:47
-
$\begingroup$ Mixtures of normals where components have unequal var's need not be normal. If OP had very small $n$'s, then hard to say what results the Shapiro-Wilk test may have given. For me, OP's key question was "Can someone please explain to me if I do also need to check the normality of residuals and if I do why I need to and how would I check this?" I gave the simplest example I could think of to answ. An added advantage of testing 40 residuals is that the test has better power than for individual tests on 10 obs. (In gen'l, a minor inconvenience is that one may need run the ANOVA to get resids.) $\endgroup$– BruceETMar 1, 2019 at 19:01
-
$\begingroup$ I'm not disagreeing with you. (: Just reminding future readers that if the variance is unequal, subtracting the mean of each group isn't sufficient. One also needs to divide by the standard deviation. I agree that your interpretation of the question is valid. $\endgroup$– knrumseyMar 1, 2019 at 19:03
-
$\begingroup$ Clarifications a good idea. One never knows what OP or other readers may have in mind. $\endgroup$– BruceETMar 1, 2019 at 19:24
-
$\begingroup$ Sorry yes I had already established that the variance of each group was the same and I then tested the normality of each group individually and found that each individual group was not normally distributed. But you’re saying I should also determine if the residuals of each group are normal by calculating the residuals of each group by subtracting the sample mean from each observation and then perform the Shapiro Wilk test? $\endgroup$– ChristieMar 1, 2019 at 19:45
One way ANOVA assumes the residuals are approximately normally distributed around their respective group means. It's not clear above when you ran the Shapiro wilk whether you ran it on the combined raw y values or separately for the y values in each group but neither is correct. In the former case we expect combined scores to become less normal as means become more different. In the latter case you are running multiple tests were you should run only one.
The correct approach is to test the normality of the combined group residuals (yi-group mean). You can use Shapiro or a qqplot for this. If results are not normal consider transforming your raw scores and retesting the residual distribution. Good luck.
