2
$\begingroup$

I am running a Friedman test for my non-Gaussian data, on MATLAB. However, I am confused about the post hoc test to choose. Two concerns:

(1) MATLAB's multcompare function's default would be Tukey HSD, but I read in multiple sites saying that it's only for parametric test (?). Link to the MATLAB guide on multcompare. If this is the case then it's quite silly that MATLAB suggested to do multiple comparison following Friedman. Other options would be Bonferroni method, Dunn and Sidák’s approach, Fisher's least significant difference procedure or Scheffé's S procedure

(2) Would it be somehow better to do a Wilcoxon test and correct to the number of hypothesis (eg in my case, 6) using Bonferonni's method?

Please if anyone could enlighten me with this

Improve this question
$\endgroup$
3
$\begingroup$

So:

  • Fisher LSD, Tukey HSD and Scheffe test are all parametric, so this is not an option for you
  • p-value corrections like Bonferroni, Dunn-Sidak and many others (Holm, Benjamini-Hochberg, ...) always can be used
  • beware of Bonferroni, it is the most conservative correction (it can dump your type-I error far below 5%)

If you are looking for post-hoc test after Friedman test, Conover and Nemenyi procedures may be options to consider.

Here you can read about them.

Two notes:

  • this is a tutorial for R pacjage called PMCMR not for MATLAB (but contains some formulas to give you some insight into those tests)
  • if you switch to R, you sholud use PMCMRplus package as PMCMR is outdated and no longer maintained
Improve this answer
$\endgroup$
  • 2
    $\begingroup$ Conover, 1999, Practical Nonparametric Statistics, 3rd, section 5.8 has a test for multiple comparisons after Friedman test. I imagine this is the same as is used in the posthoc.friedman.conover.test in the PMCMR package in R, but I didn't try to compare. It is better to use this kind of test than to use pairwise sign tests. $\endgroup$ – Sal Mangiafico Jul 18 '18 at 16:16
  • 2
    $\begingroup$ In general you are better off using a test that uses all the data, like that proposed by Conover, than to use pairwise tests that ignore most of the data for each test. For a great example of the problem with pairwise comparisons with rank based data, look up Schwenk dice. In any case, the Friedman test is a generalization of the sign test, not of the Wilcoxon signed rank test. $\endgroup$ – Sal Mangiafico Jul 18 '18 at 16:21
1
$\begingroup$

If you believe your data do not satisfy the assumptions of the parametric F test for ANOVA, and decide to use Friedman procedure, it would not make sense to use a parametric approach for the post-hoc tests. Tukey HSD, Dunn and Sidák and Fisher LSD are, at least in their original version, based on the same assumptions as the F test, so they are not advised after using Friedman test and could lead to conclusions that are in total contradiction with Friedman.

If you don't have too many treatments (or levels or conditions) and therefore not too many post hoc tests, the easiest way would be to do be to compute separately for each pair of treatments a new Freidman test, and multiply its p-value by the number of pairs tested. This provides you a Bonferonni correction that is coherent with the omnibus Friedman test.

Improve this answer
$\endgroup$
  • $\begingroup$ I actually just edited my question to add an option to do a Wilcoxon test and correct using Bonferonni's method. $\endgroup$ – Sharah Jul 10 '18 at 16:41
  • 2
    $\begingroup$ It is better to use a post-hoc test that uses all the data than pairwise tests that ignore most of the data at each step. $\endgroup$ – Sal Mangiafico Jul 18 '18 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.