1
$\begingroup$

In relation to the disorder I'm studying Screen A is reported as having a sensitivity of 90% and a specificity of 89%. Screen B is reported as having a AUC of .79 with no other data provided. Could someone please explain the relative difference between these screens? I assume screen A is more useful but but by how much?

$\endgroup$
1
  • $\begingroup$ What Frank Harrell says is a valid criticism if you have the probability scores. I get the feeling that you're getting something like a physician's diagnosis of sick/healthy, so just the discrete outcome, not a probability. Is this correct? $\endgroup$
    – Dave
    Nov 4 '20 at 19:24
0
$\begingroup$

It is not possible to conclude much from these very limited data. For screen A you only have a point estimate at a given sensitivity/specificity combination. For screen B you have a summary measure for a ROC-curve of many Spec/Sens combinations. These measures are incomparable.

It also depends on what you see as the most "useful" screen, e.g. how many false positives/false negatives you are willing to tolerate. The two screens could be desirable under different requirements.

$\endgroup$
5
  • 1
    $\begingroup$ Sens and spec require the use of arbitrary cutpoints an are usually discouraged. AUROC can be thought of as the $c$-index (concordance probability) and requires no cutoffs. But also consider proper probability accuracy scores such as Brier. $\endgroup$ Sep 2 '17 at 13:01
  • $\begingroup$ @FrankHarrell in a concrete setting where you can attach (economic) losses to type I and type II errors it can be a good idea to fix either of them and optimize the other $\endgroup$
    – Knarpie
    Sep 3 '17 at 9:19
  • 1
    $\begingroup$ Not in the least. You can't attach a loss function to backwards time-order risks. Minimizing expected losses comes from multiplying costs by direct probabilities of outcomes. $\endgroup$ Sep 3 '17 at 13:16
  • $\begingroup$ @FrankHarrell What about the positive and negative predictive value you mention here? Those aren’t direct probabilities of outcomes, but the timing is in the correct order. $\endgroup$
    – Dave
    Jul 4 '21 at 17:39
  • $\begingroup$ Yes in the right direction just not sufficiently conditional on covariates. $\endgroup$ Jul 5 '21 at 17:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.