3
$\begingroup$

@cbeleites mentions in response to the question

How to determine best cutoff point and its confidence interval using ROC curve in R?

that if one could specify the relationship between a false negative and false positive (e.g. a false negative is 10 x as bad as a false positive) that would give you a modification of the closest point to the ideal corner.

Is there a mathematical way to find that value? If so can you point me in the direction of finding it?

$\endgroup$
6
$\begingroup$

Do you mean a mathematical way to find the value of 10 that you give in the example?

No, I don't think there is, unless you can apply a cost to them, and then you find the value that equalizes the cost. E.g. if a false negative costs 10 times more than a false positive, you choose 10.

During wartime, it was someone's job to keep watch for torpedoes fired at a ship. If they saw a torpedo coming, they'd warn everyone, try to steer the ship to avoid it, and get ready to evacuate. Trouble is, a porpoise looks a lot like a torpedo. The price of a false negative is that your ship is more likely to be hit by a torpedo, and when it is hit, it's more serious. The price of a false positive is that everyone gets woken up, gets out of bed, and probably performs less well the next day. What's the relative cost of each of those? A false negative is obviously pretty bad, but how much worse than a false positive is it?

If you're charged with a crime you didn't commit, and you're convicted, a false positive is bad. How much worse is that than a false negative - i.e. letting go the bad guy? Most concede that it's worse, you don't want to lock people innocent people up, but again, it's hard to put a number on it, and pretty much impossible to calculate.

$\endgroup$
  • 4
    $\begingroup$ Well put. Seeking a cutoff (unless utilities are trivial) is equivalent to failing to understand the problem. $\endgroup$ – Frank Harrell Sep 20 '13 at 20:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.