I am trying to identify significant differences between groups from normal and non-normal distributions. When the distribution is normal the workflow is pretty straightforward: ANOVA and after that a Tukey's HSD test in order to identify the groups which are responsible of the significant variation. When the distribution is non-normal I apply a Kruskal-Wallis test and after that a pairwise Wilcoxon test (R function pairwise.wilcox.test()) to identify the groups. I know that Wilcoxon test is the non-parametric version of the t test, and is not corresponding to the Tukey's HSD test, but I want to know if what I am doing is wrong and which is the main difference between the outcomes of a t-test and a Tukey's test since they both give differences between groups.

  • $\begingroup$ I see you have the paired comparison tag. Are you confusing the term "pairwise" (comparing one sample with a second when there are more than one, perhaps as a post-hoc comparison) with the term "paired" (where the individual observations are related across samples)? They're quite different! $\endgroup$ – Glen_b Sep 9 '14 at 1:31
  • 1
    $\begingroup$ @Glen_b My bad! I used to use non-parametric, and something I read had me feeling I was making a mistake in doing so. Potato, pota... wait that doesn't work in the written word. :) (revised now, too). $\endgroup$ – Alexis Sep 9 '14 at 2:17

No, it is not a valid nonparametric alternative.

The rank sum test (either original Wilcoxon flavor, or New Improved Mann-Whitney $U$ varieties):

  1. ignore the rankings used by the Kruskal-Wallis test, and
  2. do not employ pooled variance for the pairwise tests.

See, for example, Kruskal-Wallis Test and Mann-Whitney U Test. (Also the pairwise.wilcox.test seems not to have the ties adjustments that these tests do.)

The nonparametric pairwise multiple comparisons tests you are likely looking for are Dunn's test, the Conover-Iman test, or the Dwass-Steel-Crichtlow-Fligner test. I have made packages that perform Dunn's test (with options for controlling the FWER or FDR) freely available I have implemented Dunn's test for Stata and for R, and have implemented the Conover-Iman test for Stata and for R.

Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.

Crichtlow, D. E. and Fligner, M. A. (1991). On distribution-free multiple comparisons in the one-way analysis of variance. Communications in Statistics—Theory and Methods, 20(1):127.

Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.