2
$\begingroup$

When given an example confusion matrix:

TP = 5000 FP = 1000

FN = 0 TN = 100

This would result in a point on the PR-curve where the x coordinate is fixed [1, y]

--> thus perfect recall --> meaning zero false negatives

--> however this says nothing about true negatives and false positives


When given an example confusion matrix:

TP = 5000 FP = 0

FN = 10 TN = 100

This would result in a point on the PR-curve where the y coordinate is fixed [x, 1]

--> thus perfect precision --> meaning no false positives

--> however this says nothing about false negatives and true negatives


When analysing a ROC curve it's intuitive to maximise TPR and minimise FPR. But for the PR-curve one needs to maximise both. How does one know to maximise recall vs maximising precision? When is recall more important than precision? How should you trade-off these 2 metrics?

Recall: TP/(TP+FN) How can you assume anything regarding this metric when you are interested in finding TN's?

Precision: TP/(TP+FP) Is it ok to assume precision is less important when you want to find TN's and thus 'value' points that are on the top and bottom left less? --> because you don't mind having false positives as long as you find true negatives.

$\endgroup$
1
$\begingroup$

There is no universal answer for how to optimize and it depends on the specific implementation and in particular, what are the consequences of FN vs. those of FP.

For example, the cost of an ad on a web site may be lower than that of making a marketing phone call. Hence, in the latter case you may require a higher precision when deciding whether to make a marketing call than when deciding whether to place an ad.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.