I have a data set with three groups, one of which is a control and the other two groups have been administered either a moderate or a high dose of a drug. The data are not normally or log-normally distributed, hence I am using the Kurskal-Wallis test to find if there is a difference among the three. Next, I would like to perform a post-hoc (the dosages were not planned ahead and the reason behind this is kind of complicated; however, I'm not sure if this warrants the use of the term "post-hoc" or not) analysis of the two dosages to the control (essentially, a non-parametric analog of Dunnett's test [1]).

I understand that I can run a multiple comparison test (such as MATLAB's "multcomp") to answer my next questions. However, I'm not sure which correction procedure is most suitable to use on my data. The two procedures that I suspect are the most relevant are Tukey's HSD and Dunn-Sidak.

My understanding is that there are no good multiple comparison tests that do not assume normality, though the ones that do are generally robust to the assumption of normality ([2] Ch. 4).

On the other hand, since I only need 2 of the 3 possible comparisons, I thought I could use Dunn's procedure and again according to [2] this could have a higher power than HSD if fewer comparisons that there are possible are desired.

How would one choose between Tukey's HSD and Dunn-Sidak in general? More specifically, how about in my design?

Finally, in [3], there is a section (§ 11.5.a) that introduces what's called "Dunn's test" therein (and the reference cited is [4]). Is this the same as the Dunn-Sidak correction procedure? If not, should I use that instead? If so, will this not run into the problems mentioned in Post-hoc tests after Kruskal-Wallis: Dunn's test or Bonferroni corrected Mann-Whitney tests? ?

Thank you.

[1] C. W. Dunnett (1955). A Multiple Comparison Procedure for Comparing Several Treatments with a Control, J Amer Statist Assoc, 50: 1096-1121.

[2] L. E. Toothaker, "Multiple Comparison Procedures", 1993.

[3] J. H. Zar, "Biostatistical Analysis", 2010.

[4] O. J. Dunn (1964). Multiple comparisons using rank sums, Technometrics 6: 241-252.

  • $\begingroup$ Since you are using Kruskal-Wallis test, you want to use a multiple comparison procedure on ranks. The procedure in the Dunn 1964 article is usually used. I know it has implementations in R. Those also sometimes make an adjustment to p-values for the multiple comparisons. I don't know if there are additional adjustments made to this procedure after the original paper. $\endgroup$ – Sal Mangiafico Jun 9 '18 at 10:08
  • $\begingroup$ And this is the same answer given in the Cross Validated question you cite. $\endgroup$ – Sal Mangiafico Jun 9 '18 at 10:12
  • $\begingroup$ That answer does not concern the difference between the HSD and Dunn procedures. It only talks about doing the Dunn test after KW. I'm specifically interested in the HSD-Dunn comparison here. $\endgroup$ – Sia Jun 10 '18 at 20:15

Tukey HSD, Dunn-Sidak correction, and Dunn test are all different things, and aren't really alternatives to one another.

Tukey HSD is a method to compare among means of multiple groups, as would be common as a post-hoc test after a one-way anova. It is limited by requirements for normality and homoscedasticity. Modern statistical software has techniques with fewer restrictions, e.g. this, and this.

Dunn-Sidak, usually just called Šidák, is a method of correcting either the alpha value or the p-value of a series of hypotheses to control the familywise error rate. It can be used for any collection of (independent) hypotheses. That is, not just to compare multiple means. It's not clear in the question what the "Dunn's procedure" would be in regard to post-hoc testing. Perhaps pairwise t-tests with alpha or p corrected by Sidak.

Dunn test is a post-hoc procedure appropriate to follow a Kruskal-Wallis test. In this sense, it is analogous in use to Tukey HSD, though very different.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.