1
$\begingroup$

I am studying carbon content between land cover classes in patagonia and I want to test for statistically significant differences in my data. Since my data doesn't follow a normal distribution, I decided to use non-parametric Dunn test for the job, from package dunn.test in R $(dunn.test::dun.test())$.

I identified some strange results when analyzing the complete output (all classes included), such as Impervious Land having no statistically significant differences in carbon content from Forestry Plantations. When I tested for differences for that pair of classes alone using the same method I got a statistically significant difference.

I am wondering, why do I get different adjusted p-values for a specific class combination when including all possible class combinations from when I include only the target classes?

Here is my data and the code and output I am recieving.

data <- https://drive.google.com/file/d/130rpLNaAh8zyfoTXFrCQK0xgBgV9Te3Y/view?usp=sharing

$C$ stands for the Carbon content while $clase$ stands for the land cover class. The focus classes are $clase=10 (forestry)$ and $clase=18 (impervious land)$, and this is a qqplot of the data QQplot if Carbon content

s <- fread("muestra_C_manonegra_.csv")

> dunn.test(s$C,
            g=s$clase,
            method="bonferroni")

  Kruskal-Wallis rank sum test

data: x and group
Kruskal-Wallis chi-squared = 6757.3476, df = 8, p-value = 0


                           Comparison of x by group                            
                                 (Bonferroni)                                  
Col Mean-|
Row Mean |         10         11         12         14         15         18
---------+------------------------------------------------------------------
      11 |   7.906337
         |    0.0000*
         |
      12 |   3.085390  -13.60748
         |     0.0366    0.0000*
         |
      14 |  -11.64081  -66.35126  -54.52954
         |    0.0000*    0.0000*    0.0000*
         |
      15 |   5.752852   1.546561   4.737959   14.32322
         |    0.0000*     1.0000    0.0000*    0.0000*
         |
      18 |   1.969648   0.157549   1.296411   4.679759  -0.374696
         |     0.8798     1.0000     1.0000    0.0001*     1.0000
         |
       3 |   3.074744  -4.043655   1.076868   16.77445  -3.525968  -1.036581
         |     0.0379    0.0009*     1.0000    0.0000*    0.0076*     1.0000
         |
       7 |   5.752852   1.546561   4.737959   14.32322   0.000000   0.374696
         |    0.0000*     1.0000    0.0000*    0.0000*     1.0000     1.0000
         |
       8 |   1.458533  -2.790285  -0.074239   8.055393  -3.166657  -1.227886
         |     1.0000     0.0948     1.0000    0.0000*     0.0278     1.0000
Col Mean-|
Row Mean |          3          7
---------+----------------------
       7 |   3.525968
         |    0.0076*
         |
       8 |  -0.576703  -3.166657
         |     1.0000     0.0278

alpha = 0.05
Reject Ho if p <= alpha/2

As you can see, pvalue for combination 10-18 is 0.8798, but when I analyze only the focus classes, pvalue outcome changes and becomes statistically significant:

> dunn.test(s$C[s$clase %in% c(10,18)],
            g=s$clase[s$clase %in% c(10,18)],
            method="bonferroni")

  Kruskal-Wallis rank sum test

data: x and group
Kruskal-Wallis chi-squared = 14.1263, df = 1, p-value = 0


                           Comparison of x by group                            
                                 (Bonferroni)                                  
Col Mean-|
Row Mean |         10
---------+-----------
      18 |   3.758496
         |    0.0001*

alpha = 0.05
Reject Ho if p <= alpha/2

I know that bonferroni is a very conservative method and maybe if I chose another method, or non at all, I would get statistically significant differences (which in fact I do, results not shown), but my question is why do I get different pvalues with the same data when comparing just two classes and the whole data set.

Thank you all very much, and greetings from a very not-sufficiently-snowed Patagonia.

$\endgroup$
  • $\begingroup$ The answer to your final question is that Dunn test pools the ranks across the groups. It doesn't just do pairwise Mann-Whitney tests. In general, this is why Dunn is considered a superior test than pairwise MW tests. $\endgroup$ – Sal Mangiafico Sep 10 '19 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.