I have ordinal data from 3 related samples. First i conducted a Friedman test to know if there were differences between at least two samples. Next I conducted a Wilcoxon signed rank test to know between which of the 3 samples there were differences and i generated 3 possibles pairs. However i don't know if i need to apply Bonferroni correction and basically compare my Asymp Sig (2 tailed) with P=0.05/3 instead of 0.05, or if i should mantain a p=0.05 to know if i accept or reject my null hypotesys.

Could anyone help me to clarify this question? Thank you very much.

  • $\begingroup$ One possibility is to consider the Quade test or the Friedman test (which is more akin to the sign test than the signed rank but is also suited to multiple related samples) $\endgroup$
    – Glen_b
    Jun 30, 2017 at 10:49
  • $\begingroup$ First I conducted a Friedman test, but that only tells me that differences exist at least between two samples. I need to conduct a Post-hoc to know between which samples there are differences, right? $\endgroup$
    – sofia
    Jun 30, 2017 at 11:09
  • 1
    $\begingroup$ You should probably clarify this in your question $\endgroup$
    – Glen_b
    Jun 30, 2017 at 11:14
  • $\begingroup$ Thanks for the input. Can you help me knowing what you know now about my analysis? $\endgroup$
    – sofia
    Jun 30, 2017 at 11:29
  • $\begingroup$ I think you'll get better answers on post-hoc testing from others. $\endgroup$
    – Glen_b
    Jun 30, 2017 at 11:52

2 Answers 2


It depends on what kind of false discovery probability you wish to control.

Keeping p=0.05 means you allow 5% probability of false discovery (e.g. rejecting null hypothesis while it shouldn't be rejected) in each comparison.

Taking Bonferroni correction means that you allow 5% probability for at least one false discovery.

If the latter is what you deserve, I'd recommend using some other correction (eg. Holm), because Bonferroni is very conservative. It actally dumps false discovery probability far below 5%. But, on the other hand, with 3 comparisons only this effect is not very big.

Holm correction:

First, order your p-values in incereasing order. Let $p_1$ be the smallest and $p_n$ the largest one.

Then compare each $p_i$ with $$\frac{\alpha}{n-i+1}$$

So, in your case ($n=3$):

the smallest p-value ($p_1$) is significant if it's lower than 0.05/3,

the middle one ($p_2$) is significant if it's lower than 0.05/2,

and the largest ($p_3$) is significant if it's lower than 0.05.

Plus, some people say that if you find $p_i$ significant you should also consider all lower p-values significant. So according to them, if, for example, $p_3$ is significant, while $p_2$ is not, you should also treat $p_2$ as significant.

  • $\begingroup$ Thank you for your answer. I've never heard about Holm correction. How i can easily apply it, could you please tell me? $\endgroup$
    – sofia
    Jun 30, 2017 at 11:34
  • $\begingroup$ See edited answer $\endgroup$ Jun 30, 2017 at 12:09

There are some post-hoc tests devised for Quade and Friedman tests. The best place I can refer you to is the documentation and vignette for the PMCMR package in R. The methods there are simply called "Friedman – post-hoc test after Nemenyi", "Friedman – post-hoc test after Conover", and "Quade – posthoc tests". Having played with them a little bit and here, they seem to give reasonable results, but I can't speak to their statistical qualities.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.