11
$\begingroup$

In my dataset, I have five (ordinal) groups with an x-amount of measurement. Because homoscedasticity is violated, I performed the Friedman chi-square test to see if there are any statistical differences between the groups:

fried = stats.friedmanchisquare(*[grp for idx, grp in df.iteritems()]))

This returned a statistical difference, but now I would like to find out between which groups the differences exist. In R there is a nice solution for this (Friedman's test and post-hoc analysis, https://www.r-statistics.com/2010/02/post-hoc-analysis-for-friedmans-test-r-code/), where they use the Wilcoxon-Nemenyi-McDonald-Thompson test, but I am unable to find one for Python.

Is there a possibility to do post-hoc analyses for the Friedman test? Alternatively, what would we be a good alternative for the Friedman test that does allow me to compare between groups, e.g. a generalized estimating equation?

Improve this question
$\endgroup$
1

3 Answers 3

Reset to default
10
$\begingroup$

I am currently looking into this issue myself; according to this paper there are a number of possibilities to perform posthoc-tests (Update: an extension regarding the use of non-parametric tests can be found here):

  • Perform the Nemenyi-test for all pairwise combinations; this is similar to the Tukey-test for ANOVA.
  • Perform the Bonferroni-Dunn-test; in this setting one compares all values to a list of control values.
  • Alternatively, one can perform step-up and step-down procedures sequentially testing hypotheses ordered by their significance. One can use Holm's step-down procedure, Hochberg's step-up procedure or Hommel's procedure.

The STAC Python library seems to include all these tests, except for Hommel's procedure.

Improve this answer
$\endgroup$
4
$\begingroup$

Complementing the other answer, since you asked about implementation of the post-hoc tests in Python: the Orange library implements the post-hoc tests (Nemenyi and Bonferroni-Dunn), including a function to draw a Critical Difference diagram [1]

http://docs.orange.biolab.si/3/data-mining-library/reference/evaluation.cd.html (see the section "CD diagram")

[1] Janez Demsar, Statistical Comparisons of Classifiers over Multiple Data Sets, 7(Jan):1–30, 2006.

Improve this answer
$\endgroup$
2
$\begingroup$

You can perform any of the following tests with scikit-posthocs package: Conover, Nemenyi, Siegel, and Miller post-hoc tests.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.