I am working with a data set that is non-parametric and has 12 treatments. I performed the Kruskal-Wallis test and got a significant $p$-value, and now I would like to conduct a multiple comparisons procedure to see which of the treatments differ significantly. There is a lot of information regarding this topic, but I haven't found anything that specifically addresses this issue. Any ideas??


You are looking for Dunn's test (or, say, the Conover-Iman test). This is very much like a set of pairwise rank sum tests, but Dunn's versions (1) accounts for the pooled variance implied by the null hypothesis, and (2) retains the ranking used to conduct the Kruskal-Wallis test. Performing garden variety Wilcoxon/Mann-Whitney rank sum tests ignores with these issues. One can, of course, perform family-wise error rate or false discovery rate corrections for multiple comparisons with Dunn's test.

Dunn's test is implemented for Stata in the package dunntest (from Stata type net describe dunntest, from(https://alexisdinno.com/stata) whilst connected to the Internet), and for R in the package dunn.test; both packages includes many multiple comparison's adjustment options. One might also perform Dunn's test in SAS using Elliott and Hynan's macro, KW_MC.

As I wrote in a related CV question: there are a few less well-known post hoc pairwise tests to follow a rejected Kruskal-Wallis, including Conover-Iman (like Dunn, but based on the t distribution, rather than the z distribution, strictly more powerful than Dunn's test, and also implemented for Stata in the package conovertest, and for R in the conover.test package), and the Dwass-Steel-Citchlow-Fligner tests.

Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.

| cite | improve this answer | |
  • $\begingroup$ can you elaborate why not using soft methods such as "fdr" ? $\endgroup$ – user4581 Mar 19 '15 at 11:01
  • $\begingroup$ @user4581 I am not sure I understand your question. First, what do you mean by "soft"? Second, what do you mean by "why not" one can certainly use false discovery rate methods to adjust for multiple comparisons with Dunn's test or the Conover-Iman test. $\endgroup$ – Alexis Mar 19 '15 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.