# Kruskal Wallis test with high type 1 error

using this code

pvalue = rep(NA, 100000)
n = c(352, 198, 170)
group = rep(1:3, n)
for (i in 1:100000){
x=c(rnbinom(352, size=0.9563, mu=2.27), rnbinom(198, size=1.0468, mu=2.27), rnbinom(170, size=1.3264, mu=2.27))
kruskal = kruskal.test(x ~ group)
pvalue[i] = kruskal$p.value} result = length(pvalue[pvalue<=0.05])/100000 result  I tried to examine the risk of type 1 error when comparing three neg. binomial distributions with same mu and different variances using the Kruskal Wallis Test. To my astonishment, I found out, that 8.3% of the 100,000 conducted tests had a p-value lower than 5%. I still remember from my statistics lessons, that non-parametric tests like the Kruskal Wallis are said to be more conservative and, if I remember correctly, less likely to lead to a type 1 error compared to parametric tests, when applied to data which violate certain assumptions like homogeneous variances or normal distribution. However, in this example, Kruskal Wallis apparently led to a higher type 1 error. I would be thankful, if someone could confirm to me, if the Kruskal Wallis Test could possibly be that anti-conservative or if my code is maybe wrong? However, the result seems logical, as different variances in the samples could lead to the conclusion, that they stem from different populations... BTW, I know there are better methods to examine the distributions at hand, I just tried out the Kruskal Wallis Test "for fun". Thank you very much in advance! added: Doing the same with ANOVA instead of Kruskal Wallis, I get a lower result than 5%: > pvalue = rep(NA, 100000) > n = c(352, 198, 170) > group = rep(1:3, n) > for (i in 1:100000){ + x=c(rnbinom(352, size=0.9563, mu=2.27), rnbinom(198, size=1.0468, mu=2.27), rnbinom(170, size=1.3264, mu=2.27)) + result = aov(x ~ factor(group)) + pvalue[i] = anova(result)[1,5]} > robust = length(pvalue[pvalue<=0.05])/100000 > robust  0.0438  Probably there is something wrong with the ANOVA-code, but I can't figure out what. I would appreciate any help! • Much better question but always test p < 0.05 not <=. – John Sep 7 '13 at 21:26 • Why do you claim that$H_0$is fulfilled for these negative binomial distributions ? They do not have the same median (I don't know exactly what is the location parameter in the$H_0\$ hypothesis of KW). – Stéphane Laurent Nov 10 '13 at 16:46

• Many thanks! I did perform the same thing with ANOVA as well with result = aov(x ~ factor(group)) pvalue[i] = anova(result)[1,5]} length(pvalue[pvalue<=0.05])/100000 (the part above is the same) and got  0.0461 as an answer. Thus, there seems to be something wrong with my ANOVA code, I'll check. – Renoir Pulitz Sep 7 '13 at 19:58