2
$\begingroup$

I used the following code to perform a Kruskal-Wallis test on a dataset, which gave me a significant p-value (5.159e-06).

However, when I then use pairwise.wilcox.test() to perform a Pairwise Wilcoxon Rank Sum Test, no p-value between groups shows significance. Same thing when I get the significance letters with multcompLetters2(), where every group gets the letter A.

I'm wondering if that may be affected by the low number of numeric values by group (<5).

Dataset

Table1 <- structure(list(Zone = c("1", "1", "1", "2", "2", "2", "3", "3", "3", "4", "4", "4", "5", "5", "5", "6",                                                                                         "6", "6", "7", "7", "7", "8", "8", "8", "9", "9", "9",                                                      "10", "10", "10", "11", "11", "11", "12", "12", "12", "13", "13",                                                                                            "13", "14", "14", "14", "15", "15", "15", "16", "16", "16",                                                                                           "17", "17", "17"), Response = c(13.352, 
14.0146, 14.1781, 5.62932, 5.80346, 5.89391, 8.44548, 8.30884, 
8.26276, 6.60561, 6.33102, 6.61262, 7.77392, 8.3299, 8.54713, 
5.93946, 6.10588, 6.04483, 11.8243, 11.9579, 11.7664, 15.9763, 
15.4358, 15.6228, 21.1587, 21.2021, 21.2392, 4.09461, 4.10504, 
4.04033, 13.2888, 13.0089, 12.843, 13.463, 13.4929, 13.4582, 
4.1468, 4.13986, 3.77537, 3.99322, 3.80697, 4.0791, 14.874, 
14.8796, 15.0262, 15.126, 14.7747, 15.1524, 8.86868, 8.80423, 
8.76666)), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, -51L))

Kruskal-Wallis Test

kruskal.test(Response ~ Zone, data = Table1)

Wilcoxon Test

wilcox <- pairwise.wilcox.test(Table1$Response, Table1$Zone, p.adjust.method = "BH")
$\endgroup$
3
  • 7
    $\begingroup$ You have three observations for each Zone and are doing a non-parametric rank test so, no matter what the data is, before any adjustment the probability of seeing full separation in a pairwise comparison when there is no distinction in the distributions is about $\frac{2}{6 \choose 2}=0.1$ and so "not significant" in every case; you need more observations per Zone to get anything useful from your pairwise rank test. $\endgroup$
    – Henry
    Oct 26, 2023 at 0:02
  • 2
    $\begingroup$ The fact that $130$ of the $136$ pairwise tests had this full separation would have been highly improbable if there are no distinctions in the distributions overall, so it is no surprise that the overall Kruskal-Wallis test did have low $p$-value. $\endgroup$
    – Henry
    Oct 26, 2023 at 0:02
  • $\begingroup$ @Henry: Can you add this as an answer, in the answer box? $\endgroup$ Oct 28, 2023 at 15:43

1 Answer 1

2
$\begingroup$

Requested from comments:

You have three observations for each of the $17$ Zones and are doing a non-parametric rank test so, no matter what the data is, before any adjustment the probability of seeing full separation in a pairwise comparison when there is no distinction in the distributions is about $\frac{2}{6 \choose 2}=0.1$ and so "not significant" in every case; you need more observations per Zone to get anything useful from your pairwise rank test.

The fact that $130$ of the ${17 \choose 2}=136$ pairwise tests had this full separation would have been highly improbable if there were no distinctions in the distributions overall, so it is no surprise that the overall Kruskal-Wallis test did have a low $p$-value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.