2
$\begingroup$

Im a surgeon who's trying to compare two diagnostic tests that are used to diagnose appendicitis. Two diagnostic tests were applied on 150 patients and the results were compared to a gold standard. I understand that the sensitivities and specificities can be compared by using a McNemars chi square test. But what about the positive and negative predictive values... how do you compare these? Any help would be appreciated! eg. enter image description here

$\endgroup$
  • $\begingroup$ possible duplicate? stats.stackexchange.com/questions/6309/… $\endgroup$ – shadowtalker Jun 17 '14 at 21:11
  • $\begingroup$ PPV and NPV are both sensitive (i.e. change) with different true prevalence of the thing being tested for. PPV gets miniscule rapidly as prevalence decreases (i.e. positives from a test with extremely high sensitivity and specificity are still likely to be false). So asking to compare PPV and NPV requires the addition of a prevalence for the comparison. $\endgroup$ – Alexis Jun 18 '14 at 4:19
  • $\begingroup$ Prevalence will be identical in this case as it is a comparative diagnostic accuracy study. $\endgroup$ – tristan Jun 18 '14 at 7:15
  • $\begingroup$ Then you still need to specify absolute prevalence to understand PPV and NPV in anything other than relative terms. $\endgroup$ – Alexis Jun 18 '14 at 13:18
  • $\begingroup$ Reverse engineering gives the prevalence as 124 or 125 out of 150 (0.83 to 2 decimal places) $\endgroup$ – tristan Jun 19 '14 at 12:01
2
$\begingroup$

UPDATE

I found a relevant paper!

Leisenring W, Alonzo T, Pepe MS. Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 2000; 56:345-51.

It's not an easy one to read, I think it would be best to get a statistician with experience of hierarchical regression models to read the paper and advise. If this is not an option open to you then I will try to find time to have a go with your data, if you can provide it in the following format:

+--+-----+-----+
|  |  D+ |  D- |
|  |B+ B-|B+ B-|
+--+-----+-----+
|A+|a  b |e  f |
|A-|c  d |g  h |
+--+-----+-----+

Where D is your reference standard and A and B are your two index tests (RIPASA and Alvarado). The full cross-tabulation is necessary.

(Previous answer)

In this example the test on the left is superior according to the diagnostic likelihood ratios (labeled PLR & NLR) which are both further from 1.

Diagnostic likelihood ratios, sensitivity and specificity are all significantly less sensitive to prevalence than PPV and NPV (which are both directly sensitive to prevalence) but they can still be affected if different prevalence in studies is due to spectrum bias.

Of course, a simple comparison of point estimates is not satisfactory - the better test might have only been tested in a small population and there could be no statistically significant difference. Also you want to know whether they were both evaluated in representative samples (using a single gate [cohort or nested case-control] design).

If you have multiple studies of each technique then a meta-analysis (using bivariate or HSROC method) may be appropriate. See chapter 10 (I think) of the Cochrane handbook for systematic reviews of diagnostic test accuracy.

Finally, it could be that the diagnostically superior test is not acceptable to the same population as the inferior test. Better tests are often more expensive and more invasive, both factors which can render such tests poor for, e.g., screening.

$\endgroup$
  • $\begingroup$ Sorry if this does not exactly answer your question. I'm on mobile app and had difficulty viewing question to begin with. My advice is general but applies in your case also. PPV and NPV should generally not be compared between tests. It might be acceptable in this specific case as both tests were conducted in the same population, but best to avoid the question altogether and focus on diagnostic likelihood ratios. $\endgroup$ – tristan Jun 18 '14 at 7:15
  • $\begingroup$ In response to the specific question about whether to use McNemar's test for PPV and NPV the answer is no because you don't have the same population compared in each test. It is also not appropriate to use simple comparison of proportions test because populations are not independent either. I will keep an eye open for a suitable test for this situation... $\endgroup$ – tristan Jul 2 '14 at 15:03
0
$\begingroup$

The paper you have mentioned is one of the best ways to compare predictive values of a diagnostic test and can be easily implemented using package DTComPair in R. A better statistic is "Weighted generalized score statistic" that can be implemented using same package but has an added advantage that it always reduces to commonly used score statistic when the samples are independent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.