Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
@cbeleites mentions in response to the question
How to determine best cutoff point and its confidence interval using ROC curve in R?
that if one could specify the relationship between a false negative and false positive (e.g. a false negative is 10 x as bad as a false positive) that would give you a modification of the closest point to the ideal corner.
Is there a mathematical way to find that value? If so can you point me in the direction of finding it?
Do you mean a mathematical way to find the value of 10 that you give in the example?
No, I don't think there is, unless you can apply a cost to them, and then you find the value that equalizes the cost. E.g. if a false negative costs 10 times more than a false positive, you choose 10.
During wartime, it was someone's job to keep watch for torpedoes fired at a ship. If they saw a torpedo coming, they'd warn everyone, try to steer the ship to avoid it, and get ready to evacuate. Trouble is, a porpoise looks a lot like a torpedo. The price of a false negative is that your ship is more likely to be hit by a torpedo, and when it is hit, it's more serious. The price of a false positive is that everyone gets woken up, gets out of bed, and probably performs less well the next day. What's the relative cost of each of those? A false negative is obviously pretty bad, but how much worse than a false positive is it?
If you're charged with a crime you didn't commit, and you're convicted, a false positive is bad. How much worse is that than a false negative - i.e. letting go the bad guy? Most concede that it's worse, you don't want to lock people innocent people up, but again, it's hard to put a number on it, and pretty much impossible to calculate.
