2
$\begingroup$

I am a junior physician, and I've run into some problems with the meta-analysis of diagnostic tests and I was hoping to get some input.

Some background: Aside from my undergraduate statistics courses I have no other formal statistics training (although I plan to in the near future), currently using RevMan5 for my analysis, but I am open to other software.

My meta analysis includes 10 papers looking at a specific diagnostic test. The quality of the papers isn't the best. They all include the sensitivity and specificity with a 95% CI, most include +LR and -LR, and some include PPV and NPV, only one study includes the true positives/true negative/false positives/false negatives. The same threshold is used in all the primary studies, so I wanted to calculate summary sensitivity/specificity using a Bivariate model, however it seems like I need the TP/TN/FP/FN for each study to do this.

This has me wondering should I just calculate a separate pooled mean for the sensitivity and specificity? If I do this this separately it won't account for the give and take between sensitivity and specificity, so it would underestimate the accuracy of the test (?). What would you do in this situation?

Thank you in advance. I am happy to provide more information if needed!

$\endgroup$
1
  • $\begingroup$ if you have the sample size for each paper, backcalculating the confounding matrix is trivial $\endgroup$ – carlo Jun 14 '20 at 12:45
2
$\begingroup$

Comment (too long for 'comment' format):

When dealing with 'prevalence', 'specificity', 'specificity', and so on. It is important to be clear to what population each probability applies. Prevalence is strictly a property of the population (although the proportion of the population infected might have been be estimated using screening test test data).

Sensitivity and specificity are properties of the test. For example,

$$\eta = \mathrm{Specificity} = P(\mathrm{Pos.\; test\; |\; Subj.\; infected} )\\ =\frac{P(\mathrm{Pos.\; test\; AND\;Subj.\; infected})} {P(\mathrm{Subj.\; infected})}.$$

So in combining specificity data, you have to look at the total number of infected subjects involved for each determination of specificity. You can't just average sensitivity determinations from two different studies--one using 100 infected subjects and one using 1000 infected subjects. If $\hat\eta_1 = \frac{92}{100} = 0.920$ and $\hat\eta_2 = \frac{893}{1000} = 0.893,$ then the combined estimate of sensitivity from the two studies is $\hat \eta_c = \frac{985}{1100} = 0.896.$

Depending on the method used to make confidence intervals for estimates of sensitivity, the CI for the first study might be $(0.847, 0.961),$ the CI for the second might be $(0.872, 0.911),$ and the CI using the combined estimate $(0.876, 0.912).$ [I have used 95% Agresti-Coull ("Plus-4") confidence intervals for consistency because one sample size is less than 1000. Computations use R. Perhaps see Wikipedia on binomial confidence intervals.]

eta.1 = 94/104;  pm = c(-1,1)
CI.1 = eta.1 + pm*1.96*sqrt(eta.1*(1-eta.1)/104);  round(CI.1,3)
[1] 0.847 0.961

eta.2 = 895/1004
CI.2 = eta.2 + pm*1.96*sqrt(eta.2*(1-eta.2)/1004);  round(CI.2,3)
[1] 0.872 0.911

eta.c = 987/1104
CI.c = eta.c + pm*1.96*sqrt(eta.c*(1-eta.c)/1104);  round(CI.c,3)
[1] 0.876 0.912

Moreover, I should point out that the terminologies 'false positive' and 'false negative' have been so often carelessly used in discussions about screening tests that one must be careful what they mean in each paper. For example, one common meaning for the proportion of false negatives is $P(\mathrm{Neg.\; test\;|\; Subj,\; infected}) = 1-\eta$ and another is $P(\mathrm{Neg.\; test\; AND\; Subj,\; infected})$ where the denominator would be all subjects (not just infected subjects).

Finally, predictive powers of positive and negative tests are simultaneously properties of the type of test used and of the population being tested. So, for each probability associated with a screening test, it is crucial to understand whether it depends on the test used, the population tested, or both. (I have used some of the terminology and notation above in a previous post about estimating prevalence from screening test data.)

$\endgroup$
1
$\begingroup$

If you have an estimate of the sensitivity and its confidence interval you should be able to back-calculate the sample size and hence the numbers you need. If the interval is symmetric they have used the normal approximation and back-calculating the number is trivial.

If the interval is asymmetric things get more interesting. In that case I suggest you do an iterative search. Choose a large sample size and a small one and for the given sensitivity calculate the two confidence intervals. One of these should be too wide, one too narrow. Now split the difference and recalculate the confidence interval for that. Carry on splitting the difference each time between the too wide and the too narrow until you recover their interval or you get one which is too wide on one side and too narrow on the other. Now you can compute the true positives and so on. Now repeat the whole rigmarole for specificity to get true negatives and so on.

It should be possible to automate this but for only a few examples it is probably not worth the effort.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.