1
$\begingroup$

I am training to learn how to perform meta-analysis using R. I conducted a meta-analysis using the metaprop function in R with the provided dataset. The goal is to compute the pooled specificity using the code snippet below:

Data <- structure(
  list(
    detection = c(
      "Fluorescence",
      "Fluorescence",
      "Fluorescence",
      "Fluorescence",
      "Fluorescence",
      "Fluorescence",
      "Fluorescence",
      "Lateral flow",
      "Lateral flow",
      "Fluorescence",
      "Fluorescence",
      "Fluorescence"
    ),
    TP = c(91, 139,
           29, 59, 32, 32, 24, 18, 80, 77, 261, 51),
    TN = c(62, 53, 20,
           20, 40, 37, 22, 26, 40, 19, 127, 27),
    FP = c(1, 0, 0, 1, 0, 2,
           3, 0, 0, 0, 6, 0),
    FN = c(25, 1, 0, 9, 3, 2, 1, 2, 27, 0, 7,
           12)
  ),
  row.names = c(NA, -12L),
  class = c("tbl_df", "tbl", "data.frame")
)

I encounter multiple problems when I wanted to use metaprop to compute pooled specificity using this code:

library(meta)
metaprop(event = TN,
         n = TN + FP,
         studlab = rownames(Data2),
         data = Data,
         sm = "PLOGIT",
         method.ci = "CP",
         subgroup = Data$detection,
         comb.fixed=FALSE,
         comb.random=TRUE,
         title = "Specificity")

This is the result:

Review:     Specificity

Number of studies: k = 12
Number of observations: o = 506
Number of events: e = 493

                     proportion           95%-CI
Random effects model     0.9829 [0.9462; 0.9947]

Quantifying heterogeneity:
 tau^2 = 0.7642; tau = 0.8742; I^2 = 0.0% [0.0%; 58.3%]; H = 1.00 [1.00; 1.55]

Test of heterogeneity:
          Q d.f. p-value
 Wald  3.80   11  0.9755
 LRT  19.52   11  0.0523

Results for subgroups (random effects model):
                                           k proportion           95%-CI  tau^2    tau    Q  I^2
detection = Fluorescence                  10     0.9761 [0.9374; 0.9911] 0.3949 0.6284 3.80 0.0%
detection = Lateral flow   2     1.0000 [0.0000; 1.0000]      0      0 0.00 0.0%

Test for subgroup differences (random effects model):
                  Q d.f. p-value
Between groups 0.00    1  0.9995

Details on meta-analytical method:
- Random intercept logistic regression model
- Maximum-likelihood estimator for tau^2
- Logit transformation
- Continuity correction of 0.5 in studies with zero cell frequencies
  (only used to calculate individual study results)
Warning messages:
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  unable to evaluate scaled gradient
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
   Hessian is numerically singular: parameters are not uniquely determined
3: In vcov.merMod(res.ML) :
  variance-covariance matrix computed from finite-difference Hessian is
not positive definite or contains NA values: falling back to var-cov estimated from RX
4: Ratio of largest to smallest sampling variance extremely large. May not be able to obtain stable results. 
5: Ratio of largest to smallest sampling variance extremely large. May not be able to obtain stable results. 

I have some questions:

Confidence Interval for Lateral Flow: The confidence interval for specificity in the "Lateral flow" subgroup is reported as [0.0000; 1.0000]. Given the absence of false positives in both studies using the "lateral flow" method, I expected a more precise interval. Why does it return (0-1)?

Interpretation of Warnings: The function generates several warning messages, and I'm unclear about their implications.

Any insights or suggestions on resolving these issues would be greatly appreciated. Thank you.

$\endgroup$
4
  • $\begingroup$ Your expectation is incorrect. Can you say why you think that you should have got a narrower confidence interval? $\endgroup$
    – mdewey
    Commented Jan 10 at 18:09
  • $\begingroup$ I thought 95% between 0 and 1 is just meaningless. Every specificity value in any test should lie somewhere within 0 and 1! I think there is a problem in the model fitting because there are a couple of warnings, but I don't know what they mean even though I searched a lot. $\endgroup$ Commented Jan 10 at 18:49
  • $\begingroup$ Most of the statistics you want to calculate don't make a lot of sense on a group with only 2 studies. $\endgroup$
    – wzbillings
    Commented Jan 11 at 15:11
  • $\begingroup$ @wzbillings I know two studies are not enough, but I have another dataset with similar groups but the dataset has false positives in all studies and I did not encounter with any errors like this. $\endgroup$ Commented Jan 12 at 15:44

1 Answer 1

1
$\begingroup$

The main problem here is that you are using an inappropriate model for the meta-analysis of diagnostic studies. Although the sensitivity and specificity are independent within study they are not necessarily so between studies (because of threshold effects). You need to use a proper bivariate model. Since you use R trying the mada package may be a good start.

See the article by Gatsonis and Paliwal for why your analysis is "not methodologically defensible"

$\endgroup$
1
  • $\begingroup$ Thank you very much. I was actually using this tutorial article and I've seen other articles using the same methodology as the one I've mentioned. For the univariate analysis they used meta and later further into analysis they will use mada for other types of analysis. But I am stuck in the first step! $\endgroup$ Commented Jan 12 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.