Which pairwise comparison tests are best to use after the Kruskal-Wallis test (Dunn, Conover-Iman or Steel-Dwass-Critchlow-Fligner test)? I have 3 sampling sites where I took 3 water samples in each site. I chose to do the Kruskal-Wallis test to find out if there is a significant difference in total nitrogen between the sites. I did not choose ANOVA, because normality and homoscedasticity were not always respected and I have a small sample size.
When, there is a significant difference between the sites, Xlstat offers me the option of using the Dunn, the Conover-Iman and the Steel-Dwass-Critchlow-Fligner test as a pairwise comparison test. Also, I can do the Bonferroni correction with the Dunn and Conover-Iman test. I would like to know which test is the most appropriate for my analysis and why?
 A: Use the generalization of the Wilcoxon-Mann-Whitney-Kruskal-Wallis tests: the proportional odds ordinal logistic model.  Once you fit the model with two indicator independent variables representing 3 groups you can test any pairwise contrasts you want.  Contrasts will be on the log odds scale.  This is all easily done with the R rms package orm and contrast.rms functions.  More details may be found in the Nonparametrics chapter in BBR.
A: It's possible to get a significant result with the Kruskal-Wallis test when you have three observations each from three sites.  For example, if site one had concentrations of (1,2,3), and the other sites had observations, respectively of, (4,5,6) and (7,8,9).
Practically though, environmental water quality data is usually quite variable, based on a whole host of factors.  Usually you need several to many observations – (say, something like, one sample per site per week for say 20 or 30 weeks, and then maybe repeated for two years) – to be able to make any reasonable conclusions.
Personally, I like Dunn (1964) test as a post-hoc to K-W.  I don't have a great justification for this opinion. It's probably the most traditional one.  It keeps the ranking of the observations from the original K-W test, and then performs an analysis similar to the original analysis.  In my experience it's relatively conservative, even without adjusting the p-values.
