Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I conducted an experiment, testing three conditions with 27 participants in a repeated measures design. Since the data is not normally distributed I used a Friedman test that reported a p-value of 0.1, a slight disappointment given how much time I spent in the lab. Being disappointed but curious, I still performed Wilcoxon's signed rank test as post-hoc analyses for the three pairs. The interesting thing is that these tests reported p-values 0.002, 0.004 and 0.25, respectively. Even after a - conservative - Bonferroni correction two of the p's are still significant.

Now, besides the ethical/scientific issues surrounding digging around in the data like that: why are the p-values of the Friedman and the Wilcoxon tests so far apart? And: how do I interpret these findings? They seem to contradict each other.

share|improve this question
You have repeated measures, but neither Friedman nor Wilcoxon pays any attention to the structure implied. The main answer to your post is that different questions get different answers; that is not a contradiction. – Nick Cox Jun 19 '13 at 10:04
The same kind of thing can happen with other post-hoc tests. – Glen_b Jun 19 '13 at 10:08
Note that the Friedman test in most situations has far less power than Wilcoxon-type tests. A generalization of the Wilcoxon test that is more suited to your situation is the mixed effects proportional odds model. But a quick and only slightly dirty approach is to use multiple Wilcoxon tests as you have done. On a more general note, even if the best available method resulted in a large P-value, the time was not wasted in the lab; it's just that many people are biased against "negative" studies. It is possible to learn a great deal from negative studies. – Frank Harrell Jun 19 '13 at 11:40
1) In Friedman, you rank within all 3 conditions at once, but in pairwise comparisons you rank only within 2 being currently compared. So the results need not to agree. 2) Even more: Friedman test is not the extension of Wilcoxon test over to 3+ conditions. Friedman is - I'd say - closer to be an "extended" sign z-test. – ttnphns Jun 19 '13 at 11:42
@xmjx be sure not to make the "absence of evidence is not evidence for absence" mistake. You would need to base you "if there is a difference" assessment on confidence intervals excluding meaningful differences, not on large P-values. – Frank Harrell Jun 19 '13 at 13:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.