I am trying to identify significant differences between groups from normal and non-normal distributions. When the distribution is normal the workflow is pretty straightforward: ANOVA and after that a Tukey's HSD test in order to identify the groups which are responsible of the significant variation. When the distribution is non-normal I apply a Kruskal-Wallis test and after that a pairwise Wilcoxon test (R function pairwise.wilcox.test()) to identify the groups. I know that Wilcoxon test is the non-parametric version of the t test, and is not corresponding to the Tukey's HSD test, but I want to know if what I am doing is wrong and which is the main difference between the outcomes of a t-test and a Tukey's test since they both give differences between groups.
-
$\begingroup$ I see you have the paired comparison tag. Are you confusing the term "pairwise" (comparing one sample with a second when there are more than one, perhaps as a post-hoc comparison) with the term "paired" (where the individual observations are related across samples)? They're quite different! $\endgroup$ – Glen_b Sep 9 '14 at 1:31
-
1$\begingroup$ @Glen_b My bad! I used to use non-parametric, and something I read had me feeling I was making a mistake in doing so. Potato, pota... wait that doesn't work in the written word. :) (revised now, too). $\endgroup$ – Alexis Sep 9 '14 at 2:17
No, it is not a valid nonparametric alternative.
The rank sum test (either original Wilcoxon flavor, or New Improved Mann-Whitney $U$ varieties):
- ignore the rankings used by the Kruskal-Wallis test, and
- do not employ pooled variance for the pairwise tests.
See, for example, Kruskal-Wallis Test and Mann-Whitney U Test. (Also the pairwise.wilcox.test seems not to have the ties adjustments that these tests do.)
The nonparametric pairwise multiple comparisons tests you are likely looking for are Dunn's test, the Conover-Iman test, or the Dwass-Steel-Crichtlow-Fligner test. I have made packages that perform Dunn's test (with options for controlling the FWER or FDR) freely available I have implemented Dunn's test for Stata and for R, and have implemented the Conover-Iman test for Stata and for R.
References
Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.
Crichtlow, D. E. and Fligner, M. A. (1991). On distribution-free multiple comparisons in the one-way analysis of variance. Communications in Statistics—Theory and Methods, 20(1):127.
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.
