I recently conducted research where I have chosen the Friedman nonparametric test to check if there are significant differences in dependent variable values obtained on 4 different occasions. I find it strange that Friedman test is not significant. Subsequently, I tested pairwise with the Wilcoxon signed-rank test and i got 2 pairs statistically significant. Please explain!
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Note that the Friedman is very like a multisample analogue of the sign test, not the signed rank. Leaving that aside, you still don't always expect all possible pairwise comparisons to correspond exactly to the overall test -- in general across many different forms of pairwise comparison you can get significant pairwise comparisons without the test being significant overall and you can get no significant pairwise differences when you reject the omnibus test. This can happen with ANOVA for example. $\endgroup$– Glen_bCommented Aug 9, 2017 at 1:48
-
$\begingroup$ +1 For your question. Had to change my answer a bit. $\endgroup$– CarlCommented Aug 9, 2017 at 2:46
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
Did you adjust the Wilcoxon test results for multiple comparisons?
See here: https://en.wikipedia.org/wiki/Multiple_comparisons_problem for some detail on the problem with multiple comparisons. it is fairly easy to find/simulate examples where an overall test fails to find significance, but a few out of the multiple pairwise comparisons are.
answered Aug 8, 2017 at 17:23
-
$\begingroup$ Yess if i apply benjamini and hochberg i loose all the significant results😭 I tried boniferoni but my mentor disagrees because it is too conservative 😬😩 any other suggestions? $\endgroup$ Commented Aug 8, 2017 at 17:52
-
$\begingroup$ @MancaNolimal You're doing p-value hacking, and is not recommended. Please stop abusing statistics. $\endgroup$ Commented Aug 9, 2017 at 0:45
-
$\begingroup$ @smallchess i am not... ill respresent results the way they are. Even if not significant😅👌🏻 I just asked if there exist other ways of adapting results for multiple comparisons. Thanks again:) $\endgroup$ Commented Aug 9, 2017 at 1:28
$\begingroup$
$\endgroup$
3
The Friedman test compares multiple differences of location. The Wilcoxon signed-rank test tests differences of location of the data for pairwise data only, and as per @GregSnow would need correction for multiple comparisons. Location testing is like difference of means or medians but isn't quite median difference testing for the Wilcoxon test, or probably either for the Friedman test.
Also, note that different tests, even when adjusted to test for the same things, have different powers. That is, sometimes one test is better at showing a significant difference than another. We can generally get some idea of which test is better if we have enough data, or enough tests so that how good the tests are becomes obvious. The Wilcoxon signed-rank test generally has good power.
-
$\begingroup$ Carl -- can you explain the intent of your first sentence please. I don't see how the Friedman test is related to variance. $\endgroup$– Glen_bCommented Aug 9, 2017 at 1:48
-
$\begingroup$ @Carl thanks for your explanation. Very much appreciated. According to my data and my hypothesis Ill use willcoxon and then use correction for multiple comparison. $\endgroup$ Commented Aug 9, 2017 at 2:36
-
$\begingroup$ @Glen_b Actually, you are correct, it is a location test as well. It does use variance ranking a fair bit, but that seems to be only for scaling. $\endgroup$– CarlCommented Aug 9, 2017 at 2:45