# Difference in adjusted pvalues for comparisons between classes when using complete data and only two classes using dunn.test::dunn.test() in R

I am studying carbon content between land cover classes in patagonia and I want to test for statistically significant differences in my data. Since my data doesn't follow a normal distribution, I decided to use non-parametric Dunn test for the job, from package dunn.test in R $$(dunn.test::dun.test())$$.

I identified some strange results when analyzing the complete output (all classes included), such as Impervious Land having no statistically significant differences in carbon content from Forestry Plantations. When I tested for differences for that pair of classes alone using the same method I got a statistically significant difference.

I am wondering, why do I get different adjusted p-values for a specific class combination when including all possible class combinations from when I include only the target classes?

Here is my data and the code and output I am recieving.

$$C$$ stands for the Carbon content while $$clase$$ stands for the land cover class. The focus classes are $$clase=10 (forestry)$$ and $$clase=18 (impervious land)$$, and this is a qqplot of the data

s <- fread("muestra_C_manonegra_.csv")

> dunn.test(s$$C, g=s$$clase,
method="bonferroni")

Kruskal-Wallis rank sum test

data: x and group
Kruskal-Wallis chi-squared = 6757.3476, df = 8, p-value = 0

Comparison of x by group
(Bonferroni)
Col Mean-|
Row Mean |         10         11         12         14         15         18
---------+------------------------------------------------------------------
11 |   7.906337
|    0.0000*
|
12 |   3.085390  -13.60748
|     0.0366    0.0000*
|
14 |  -11.64081  -66.35126  -54.52954
|    0.0000*    0.0000*    0.0000*
|
15 |   5.752852   1.546561   4.737959   14.32322
|    0.0000*     1.0000    0.0000*    0.0000*
|
18 |   1.969648   0.157549   1.296411   4.679759  -0.374696
|     0.8798     1.0000     1.0000    0.0001*     1.0000
|
3 |   3.074744  -4.043655   1.076868   16.77445  -3.525968  -1.036581
|     0.0379    0.0009*     1.0000    0.0000*    0.0076*     1.0000
|
7 |   5.752852   1.546561   4.737959   14.32322   0.000000   0.374696
|    0.0000*     1.0000    0.0000*    0.0000*     1.0000     1.0000
|
8 |   1.458533  -2.790285  -0.074239   8.055393  -3.166657  -1.227886
|     1.0000     0.0948     1.0000    0.0000*     0.0278     1.0000
Col Mean-|
Row Mean |          3          7
---------+----------------------
7 |   3.525968
|    0.0076*
|
8 |  -0.576703  -3.166657
|     1.0000     0.0278

alpha = 0.05
Reject Ho if p <= alpha/2


As you can see, pvalue for combination 10-18 is 0.8798, but when I analyze only the focus classes, pvalue outcome changes and becomes statistically significant:

> dunn.test(s$$C[s$$clase %in% c(10,18)],
g=s$$clase[s$$clase %in% c(10,18)],
method="bonferroni")

Kruskal-Wallis rank sum test

data: x and group
Kruskal-Wallis chi-squared = 14.1263, df = 1, p-value = 0

Comparison of x by group
(Bonferroni)
Col Mean-|
Row Mean |         10
---------+-----------
18 |   3.758496
|    0.0001*

alpha = 0.05
Reject Ho if p <= alpha/2


I know that bonferroni is a very conservative method and maybe if I chose another method, or non at all, I would get statistically significant differences (which in fact I do, results not shown), but my question is why do I get different pvalues with the same data when comparing just two classes and the whole data set.

Thank you all very much, and greetings from a very not-sufficiently-snowed Patagonia.

• The answer to your final question is that Dunn test pools the ranks across the groups. It doesn't just do pairwise Mann-Whitney tests. In general, this is why Dunn is considered a superior test than pairwise MW tests. – Sal Mangiafico Sep 10 '19 at 8:18