I have a dataset were people sampled litter at 12 different locations. As the data does not have homogeneous variances I performed a Kruskal Wallis rank sum test (which came up significant) and followed up with Dunn's test. Dunn's test now compares each of the different locations against each other - a total of 66 comparisons - and gives me an unadjusted and an adjusted p-value. Am I correct in assuming that the adjusted p-values adjusts for the amount of comparisons done? Or is there a limit on how many comparisons can be meaningful done by Dunn's test?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
is there a limit on how many comparisons can be meaningful done by Dunn's test?
There is not a hard limit, no—though smaller differences will start to become insignificant when performing (and adjusting for) more tests, as I understand it.
Am I correct in assuming that the adjusted p-values adjusts for the amount of comparisons done?
I don't have enough reputation to make a comment on your question, but we have no way of knowing which adjustments are being made without more information about how you're executing Dunn's test. Are you using R? The "dunn.test" package, for example, has multiple options to account for multiple tests, but the default is to report only unadjusted p-values. If it's already reporting adjusted values though, I have to assume it's to account for multiple tests.
Bonferroni correction has been the most popular method to do this, per "Nonparametric Pairwise Multiple Comparisons in Independent Groups Using Dunn’s Test" (Dinno, 2015), but there have been improvements in the 50+ years since it was developed. The documentation for whatever software you're using should specify which one it's using.
