0
$\begingroup$

I've asked the same question elsewhere.


These two articles recommend Dunn's test as non-parametric post hoc multiple comparison test following Kruskal-Wallis test.

How the Dunn method for nonparametric comparisons works

https://www.graphpad.com/guides/prism/7/statistics/stat_how_the_dunn_method_for_nonpar.htm?toc=0

Dunn's test

https://stats.stackexchange.com/tags/dunn-test/info

MATLAB's multcompare function offers a few choices for post hoc tests, including 'dunn-sidak'. The name suggests that 'dunn-sidak' is Dunn's test mentioned above, but the description here says that it uses t distribution, so I thought that it is unlikely that this approach is used for non-paramatric data.

Is so-called Dunn's test identical to Dunn & Sidák’s Approach used by MATLAB multcompare?

If not, is there a way to run Dunn's test with MATLAB? Perhaps, I need to use R (it has a package dunn.test) or Python instead?

$\endgroup$
  • 1
    $\begingroup$ Dunn's test is based on a z test statistic, not a t test. "Šidak" refers to a variation of Dunn's "Bonferroni" adjustment for multiple comparisons (instead of adjusting $\alpha$ by dividing it by $m$ comparisons, the Šidak adjustment is given by $1 - (1-\alpha)^{\frac{1}{m}}$). tl;dr: Dunn wrote both a test (Dunn's test), and motivated familywise error correction procedures, of which the Šidak adjustment is one. $\endgroup$ – Alexis Feb 11 at 15:19
  • 1
    $\begingroup$ This link appears to contain the Dunn 1964 test for MatLab. I didn't test it. $\endgroup$ – Sal Mangiafico Feb 11 at 23:10
  • $\begingroup$ Thanks for the comments, guys. So, they're different. How confusing! Also thanks for the MATLAB function. $\endgroup$ – Kouichi C. Nakamura Feb 12 at 16:50
  • $\begingroup$ I've got a reply from MathWorks (see below), and their (his) conclusion is that 'dunn-sidak' option of multcompare is identical to so-called 'Dunn's test', thought further testing may be ideal. $\endgroup$ – Kouichi C. Nakamura Feb 17 at 12:22
0
$\begingroup$

I've got a reply from MathWorks about this. Despite my and others' suspect that 'dunn-sidak' and so called Dunn's test are different (see above comments), their (his) conclusion is that they are the same.

It would be worth comparing the same example data with R and MATLAB and see if they match.


Hello Kouichi,

I am writing in reference to your Technical Support Case ********** regarding 'Is multcompare's 'dunn-sidak' option identical to so-called Dunn's test?'.

I looked at the original Dunn paper and I see equations (4) and (5) defining mean ranks and their variances. Those expressions match the code I see in:

  • anova1.m - look for the place where 'kruskalwallis' is assigned into a struct field, at line 268 in the latest release
  • multcompare.m - look for the case 'kruskalwallis' block

You can open the files by simply typing "edit" in MATLAB. For example:

>> edit anova1.m

Hence, it looks like they are computing the same test statistics. Furthermore, by looking at the R package for this that you mentioned, I see that it offers a variety of multiple comparison approaches:

"none", "bonferroni", "sidak", "holm", "hs", "hochberg", "bh", "by"

which are quite similar to the ones from "multcompare.m"; i.e.:

'tukey-kramer' 'dunn-sidak', 'bonferroni', 'scheffe', 'hsd', 'lsd' (same as 'none')

So I would conclude that you can use "kruskalwallis.m" and "multcompare.m" to carry out Dunn's test.

Having said that, if you find a published example worked out in a book or paper, it may be sensible to try that out in MATLAB and see if it matches the desired test. If you do not know how to, it would be great if you cold provide us with a link and we can see if we duplicate the same test with the existing functions.

Regarding the "Steel-Dwass" test, I will put an enhancement request on your behalf so our development team can consider adding it on a future MATLAB release.

$\endgroup$
  • 1
    $\begingroup$ Yes, I think it's (always) a good idea to compare the results from a software function to a published example to see if it's doing what you think it is. There are different statistical procedures that have similar names, and also there are variants of the same procedure. Also, documentation for software isn't always very clear. $\endgroup$ – Sal Mangiafico Feb 17 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.