3
$\begingroup$

What happens with sens/spec and PPV/NPV if one performs a test twice (or three times etc.)?

I will give an example from my own clinical practice. At many hospitals we have a problem with multiresistant bacteria, e.g. MRSA. Classic tests for whether the patient has MRSA or not take three days before results. In my clinical practice, Intensive Care, every single day costs tremendous amounts of money and every saved day would be valuable. There are now some quick tests but arguments has been raised against their spec/sens.

Please help me understand what would happen if we perform a test twice. We have one test with the following performance:
Sensitivity 94.6%
Specificity 96.9%
NPV 99.9
The prevalence of MRSA in this particlar population could be around 1%

In practice, we are looking for those with negative results. These patients would go through an additional test to "make sure" they are negative, while those testing positive would not be sent to the ward.

$\endgroup$
3
$\begingroup$

A big problem in trying to answer this question is that it depends on the nature of false-negative and false-positive test results. Are these due to technical errors within the test, or to patient-specific issues?

If one patient has a strain of MRSA that does not register in the test, and another patient has some other less-serious infection that shows up as MRSA in the test, then no test/re-test strategy will help. Absent technical errors within the test, the former will always have a false-negative result and the latter a false-positive. If such types of patient samples are a major source of false test results then you cannot, say, "make sure" that a particular patient is negative by re-testing.

If such patient-specific issues can be ruled out (and that might be a big "if"), you could try to develop testing strategies based on $P(Test^+|MRSA^+)$ (sensitivity, 0.946 here) and $P(Test^-|MRSA^-)$ (specificity, 0.969 here) in simple binomial models. Work directly with the probabilities and the expected MRSA prevalence rather than with the (often confusing) sensitivity/specificity/NPV... terminology. For example, if the quick test is inexpensive and all false results come from test technical errors, you could consider running the test, say, in triplicate on all patients at the start and accepting the result of 2 or more tests having the same result.

Such a strategy must be developed, however, in combination with the costs of different types of errors: the cost of keeping an MRSA-negative patient in the ICU and the cost of releasing an MRSA-positive patient into the ward. Specificity and sensitivity are somewhat hollow concepts without an associated cost/benefit analysis.

The situation you describe illustrates these tradeoffs even if there are only technical rather than patient-specific test errors. Your present screening strategy keeps a patient in the ICU if there is one positive test out of a maximum of 2 trials. Out of 100,000 patients including 1000 that are MRSA-positive, a simple binomial model suggests that this strategy keeps 6043 MRSA-negative patients unnecessarily in the ICU while releasing 3 MRSA-positive patients to the ward.

If you instead performed 3 tests and kept those with at least 2 positive test results in the ICU, binomial statistics indicate that you would only keep 280 MRSA-negative patients in the ICU but would release 8 MRSA-positive to the ward. Do the savings from 5763 fewer unnecessary ICU cases outweigh the costs of 5 extra releases of MRSA-positive patients to the ward plus the costs of the extra tests? Specificity and sensitivity alone cannot tell you which decision to make.

Finally, as @FrankHarrell points out, other clinical signs must be factored in. If a patient with an MRSA-positive quick-test result has been responding to a conventional antibiotic, which result are you going to believe?

$\endgroup$
5
$\begingroup$

You are conceptualizing this as considering sens and spec to be not conditioned on any other information when in fact sens and spec change with knowledge about other patient characteristics or previous tests. More about reasons sens and spec are not constant and in general are not as useful as clinicians think may be found in Chapter 18 of http://biostat.mc.vanderbilt.edu/tmp/bbr.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.