10
$\begingroup$

For a confusion matrix, there are a variety of useful rates, ratios and indices. But I cannot find the one I care about:

FP / (FP + FN)

Of course this measure is not defined for a perfect classifier, one in which both false positives and false negatives are zero. But for everything short of perfect, this measure is useful as it shows whether false positives or false negatives dominate.

Is there a name for this measure? Perhaps I missed how it is algebraically equivalent to one of the other measures.

$\endgroup$

3 Answers 3

7
$\begingroup$

I would call this the proportion of the misclassifications/mistakes that are false positives.

The denominator is the total number of misclassifications/mistakes. Of these mistakes, some are false positives and some are false negatives; the particular numerator here considers the number of false positives.

Hence, dividing the number of false positives in particular by the total number of mistakes gives the proportion of mistakes that are false positives.

I do not know this to be one of the common confusion matrix statistics like sensitivity and specificity are, but I struggle to think of a term that provides more clarity about meaning than my proposed term.

$\endgroup$
7
$\begingroup$

I never heard the name for it and Wikipedia lists most of the named metrics like this, so my guess would be that it does not have a name.

$\endgroup$
1
  • 2
    $\begingroup$ Amazingly, not one of these has denominator $\mbox{FP}+\mbox{FN}$. $\endgroup$
    – J.G.
    Jul 19, 2023 at 8:27
1
$\begingroup$

But for everything short of perfect, this measure is useful as it shows whether false positives or false negatives dominate.

What would it be useful for and do you need that specific ratio for anything?

Because unlike many of the other rates, ratios and indexes, this one isn't really very suitable for tracking, comparing and optimizing towards the goal of increasing TP or reducing FP or FN.

Instead this only looks at the ratio of FP and FN. And to be honest it also looks like a rather complicated way of doing it. So if you're just interested in what number is bigger... Well look at the two numbers and it's right there. Or why not just take the ratio FN/FP or FP/FN which already tells you the factor of difference.

But most importantly that ratio is usually not of any significant interest of it's own. Like primarily your focus would be to increase accuracy and to reduce the overall misclassification and after you've optimized that to a comfortable extend you'd usually have a preference for one kind of error.

So idk suppose you make a test for a disease then false positives are better than false negatives, as you'd rather expend more care on people who don't need it than to have ill people run around and further spread the disease. While for example if you're producing billions of screws per day you're preference might be towards minimizing false positives because it might be less costly to discard perfectly good screws than to pay for the damages caused by faulty screws.

It's rather rare that your priority would be to regulate a specific ratio between two kinds of misclassification. Especially when this really just regulates the ratio of errors and not the overall prevalence of errors. So let's say you want a ration of 60% FP and 40% FN, but you have 40% FP and 60% FN. Then one way to optimize would be to increase the overall amount of errors by adding false positives until the ratio fits.

That would be completely counterintuitive with regards to any real world example. So rather than trying to optimize that you'd rather look at the max(FN, FP) and then try to improve the respective sensitivity or specificity score while checking the other for growth.

So even if you were interested in that ratio you wouldn't really be able to use it to track that progress, would you?

$\endgroup$
1
  • 1
    $\begingroup$ I am modeling a distribution of points on a ROC curve. If I start with a distribution on FPR and project that to the ROC, the resulting distribution is biased to the right. But if I model a distribution on the proportion of mistakes that are false positives—a distribution on the diagonal—and project that to the ROC, I end up with what the right distribution. So perhaps FP/(FP + FN) is not widely useful, but useful for my problem. $\endgroup$ Jul 20, 2023 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.