1
$\begingroup$

In the R caret documentation for confusionMatrix(): 

                        TP
sensitivity    = -----------------
                      TP + FN

                        TP
detection rate = -----------------
                 TP + FP + FN + TN

Practically, why would someone want to calculate detection rate? What is its usefulness?

Many people elsewhere think detection rate is the same as sensitivity. Why is such confusion so common?

I'm a bit confused!

Improve this question
$\endgroup$
1
  • $\begingroup$ With all due respect to Max, who authored the function and package, "Detection rate" is more than a little bit misleading. I would never use such terminology in reporting the validation of a diagnostic test. Merely, "probability test positive" or some other contextually appropriate description more than suffices. $\endgroup$
    – AdamO
    Commented Dec 18, 2017 at 17:40

2 Answers 2

Reset to default
1
$\begingroup$

One possible implication for this measure in policy and evaluation is for cost-effectiveness. Suppose, for instance, we intend to implement targeted screening for a disease in a population of healthcare subscribers receiving routine care. The current standard of care is to wait for the patient to present with symptoms of disease in clinic; however, treatment at this point tends to have a low success rate, many complications, and high cost. The rationale for screening is to deliver more timely, less invasive, and more successful treatment at a comparable cost.

If a patient is identified as a true positive, not only must they undergo treatment, but their longer survival requires them to be on treatment for a longer duration, and individuals who would have died from other disease before becoming symptomatic undergo treatment unnecessarily for the disease. The cost of the screening program must incorporate the costs of treating those identified with the disease.

Improve this answer
$\endgroup$
10
  • $\begingroup$ I thought of a similar example at first but realized it doesn't quite fit. The problem is, the cost of this screening program will be determined by the total number of patients selected by the screen for further treatment, which includes both true positives and false positives. In this case, TP+FP/N is a better measure of cost than TP/N (where N is the total number). Without another test, TPs and FPs are totally indistinguishable in practice. If we want to know how cost effective the test is, the positive predictive value appropriately measures how often the test is right for positive cases. $\endgroup$
    – Nuclear Hoagie
    Commented Dec 18, 2017 at 18:19
  • $\begingroup$ @AdamO Clever example. I would add that in order to be identified as a true positive (vs false positive) a secondary test (that is more specific than the first test and less costly than the full treatment, perhaps) would have to be administered. $\endgroup$
    – Underminer
    Commented Dec 18, 2017 at 18:21
  • $\begingroup$ @NuclearWang often times the considerations for confirmatory tests is not the cost, but rather that it is invasive. In breast cancer for instance, a screening mammogram can identify abnormal masses, but the biopsy confirms it is cancer. The cost of the biopsy among TPs and FPs may be negligible. The surgical resection, years of chemotherapy, and surveillance that takes places among the TPs incurs the most cost. $\endgroup$
    – AdamO
    Commented Dec 18, 2017 at 18:46
  • 1
    $\begingroup$ @Underminer indeed. There are four aspects to evaluating a screening program: the sensitivity and cost of the screen, the cost of the confirmation, the cost of treatment in the screen detected persons, the reduced cost of treatment in symptomatic persons (hopefully fewer go on to develop symptoms and thus be identified as diseased as a result of screening if lessons from cancer screening studies are to teach us anything.) $\endgroup$
    – AdamO
    Commented Dec 18, 2017 at 18:49
  • 1
    $\begingroup$ @AdamO I suppose this makes more sense in the context of a confirmatory secondary test. In effect, though, this artificially sets the FPR of the original test to 0. In the breast cancer example, you assume that the biopsy test has 0 FPs so that no healthy people undergo surgery/chemo/etc. Without that perfect secondary test, some healthy people will be treated at exactly the same cost as treating someone with cancer. My point is that you will never know who is TP and who is FP unless you have that second test, and at that point we're no longer discussing metrics of the original test. $\endgroup$
    – Nuclear Hoagie
    Commented Dec 18, 2017 at 19:15
1
$\begingroup$

It is a parameter that will vary according to the dataset. If you get a high overall performance on a moderate detection rate, it would make me wonder how the performance would vary on a smaller dataset with more positives.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.