1
$\begingroup$

I would like to calculate heterogeneity statistics (Cochran's Q, I-squared) for a meta-analysis of diagnostic accuracy. I successfully used the 'mada' package in R to calculate these statistics, but unfortunately I don't think output is accurate. My goal is to calculate heterogeneity statistics for positive and negative likelihood ratios (LR+, LR-).

The contingency tables below come from a published meta-analysis, which I am using to test the accuracy of my calculations. The problem is that I have so far been unable to replicate the I-squared and Q values that the authors of this meta-analysis found.

> data_DH
                                        study TP FP FN TN
1                    Girbau & Schwartz (2008)  9  1  2 10
2         Gutierrez-Clellen & DeCurtis (1999)  4  0  4  9
3             Gutierrez-Clellen et al. (2006)  7  0  1  8
4             Gutierrez-Clellen et al. (2006)  7  0  0  7
5             Gutierrez-Clellen et al. (2006)  3  0  4  7
6  Gutierrez-Clellen & Simon-Cereijido (2007)  8  1  2  9
7             Gutierrez-Clellen et al. (2006) 10  6  1 10
8             Gutierrez-Clellen et al. (2006)  1  0 10 16
9                  Jacobson & Schwartz (2005) 11  2  1 13
10                 Jacobson & Schwartz (2005) 11  4  1 11
11                 Jacobson & Schwartz (2005) 12  4  0 11
12                            Restrepo (1998)  7  0  1  8
13                            Restrepo (1998)  7  0  1  8
14                  Roseberry & Connel (1991) 10  1  3 12
15 Simon-Cereijido & Gutierrez-Clellen (2007)  4  1  1  4
16 Simon-Cereijido & Gutierrez-Clellen (2007)  4  1  1  4
17 Simon-Cereijido & Gutierrez-Clellen (2007)  5  1  0  4

In the code below I am using the 'madauni' function in the 'mada' package to calculate heterogeneity statistics for positive and negative likelihood ratios in the next block of code. Although my LR+ heterogeneity statistics are in the ballpark of what I expected, based on what was published by the original authors, the LR- values are totally different.

Is there something obviously wrong with the code? Are there are any other recommendations to calculate these statistics? I have played around with a few different meta-analysis packages, but I continue to get numbers that seem off. For those who have some experience in diagnostic accuracy meta-analysis, I welcome any and all recommendations. Thank you!!!

Expected values (from original meta-analysis)

LR+: I-squared = 0%, Cochran's Q = 11.97

LR-: I-squared = 86.7%, Cochran's Q = 119.93

library(mada)

### Positive likelihood ratio (LR+) heterogeneity
> summary(mada_posLR)
Call:
madauni(x = data_DH, type = "posLR", method = "DSL", 
    correction = 0.5, correction.control = "single")

Estimates:
        DSL estimate 2.5 % 97.5 %
posLR          4.121 2.937  5.783
lnposLR        1.416 1.077  1.755
tau^2          0.000 0.000  0.183
tau            0.000 0.000  0.427

Cochran's Q: 9.493 (16 df, p = 0.892)
Higgins' I^2: 0%


### Negative likelihood ratio (LR-) heterogeneity
> summary(mada_negLR)
Call:
madauni(x = data_DH, type = "negLR", method = "DSL", 
    correction = 0.5, correction.control = "single")

Estimates:
        DSL estimate  2.5 % 97.5 %
negLR          0.263  0.163  0.424
lnnegLR       -1.337 -1.815 -0.859
tau^2          0.476  0.000  0.282
tau            0.690  0.000  0.531

Cochran's Q: 9.284 (16 df, p = 0.901)
Higgins' I^2: 0%

```
$\endgroup$
2
  • $\begingroup$ Rather more puzzling is that the confidence interval for $\tau$ does not include the DSL estimate. I think this can happen with sparse data but I am not sure. If you do not get any answers here try registering and posting on the R meta-analysis mailing list stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis/ where many of the package authors post. $\endgroup$ – mdewey Sep 27 '19 at 16:39
  • $\begingroup$ Puzzling, indeed. Perhaps the small number of studies, and the relatively small sample sizes within each, does have something to do with this. $\endgroup$ – José A. Ortiz Sep 28 '19 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.