Can Mann-Whitney test be used for post-hoc comparisons after Kruskal-Wallis?

Question

I have a simulation where an animal is placed in a hostile environment and timed to see how long it can survive using some approach to survival. There are three approaches it can use to survive. I ran 300 simulations of the animal using each survival approach. All simulations take place in the same environment but there's some randomness so it's different each time. I time how many seconds the animal survives in each simulation. Living longer is better. My data looks like this:

Approach 1, Approach 2, Approach 2
45,79,38
48,32,24
85,108,44
... 300 rows of these

I'm unsure of everything I do after this point so let me know if I'm doing something stupid and wrong. I'm trying to find out if there's a statistical difference on lifespan using a particular approach.

I ran a Shapiro test on each of the samples and they came back with tiny p values, so I believe the data isn't normalized.

Data on rows have no relationship to each other. The random seed used for each simulation was different. As a result, I believe the data isn't paired.

Because the data is not normalized, not paired and there were more than two samples, I ran a Kruskal Wallis test which came back with a p-value of 0.048. I then moved on to a post hoc, selecting Mann Whitney. In really not sure if Mann Whitney should be used here.

I compared each survival approach with each other approach by performing the Mann Whitney test i.e. {(approach 1, approach 2), (approach 1, approach 3), (approach 2, approach 3)}. There was no finding of statistical significance between the pair (approach 2, approach 3) using a two tailed test but there was significance difference found using a one tailed test.

Problems:

1. I don't know if using Mann Whitney like this makes sense.
2. I don't know if I should be using a one or two tailed Mann Whitney.

Answer 1

No, you should not use the Mann-Whitney $U$ test in this circumstance.

Here's why: Dunn's test is an appropriate post hoc test^* following rejection of a Kruskal-Wallis test. If one proceeds by moving from a rejection of Kruskal-Wallis to performing ordinary pair-wise rank sum (i.e. Wilcoxon or Mann-Whitney) tests, then two problems obtain: (1) the ranks used for the pair-wise rank sum tests are not the ranks used by the Kruskal-Wallis test; and (2) the rank sum tests do not use the pooled variance implied by the Kruskal-Wallis null hypothesis. Dunn's test does not have these problems

Post hoc tests following rejection of a Kruskal-Wallis test which have been adjusted for multiple comparisons may fail to reject all pairwise tests for a given family-wise error rate or given false discovery rate corresponding to a given $\alpha$ for the omnibus test, just as with any other multiple comparison omnibus/post hoc testing scenario.

Unless you have reason to believe that one group's survival time is longer or shorter than another's a priori, you should be using the two-sided tests.

Dunn's test can be performed in Stata using dunntest (type net describe dunntest, from(https://www.alexisdinno.com/stata)), and in R using the dunn.test package.

Also, I wonder if you might take a survival analysis approach to assessing whether and when an animal dies based on different conditions?

^* A few less well-known post hoc pair-wise tests to follow a rejected Kruskal-Wallis, include Conover-Iman (like Dunn, but based on the t distribution, rather than the z distribution, implemented for Stata in the conovertest package, and for R in the conover.test package), and the Dwass-Steel-Citchlow-Fligner tests.

Answer 2

A unifying generalization of Kruskal-Wallis/Wilcoxon is the proportional odds model, which admits general contrasts with either pointwise or simultaneous confidence intervals for odds ratios. This is implemented in my R rms package's orm and contrast.rms functions.

Answer 3

You can also use the critical difference after Conover or the critical difference after Schaich and Hamerle. The former is more liberal whereas the latter is exact but lacks a bit of power. Both methods are illustrated on my website brightstat.com and brightstat's webapp also lets you calculate these critical differences and perform the post-hoc tests right away. Kruskal-Wallis on brightstat.com

Answer 4

If you are using SPSS, do the post-hoc Mann-Whitney with Bonferroni correction (p value divided by the number of groups).