What are correct values for precision and recall in edge cases?

Question

Precision is defined as:

p = true positives / (true positives + false positives)

Is it correct that, as true positives and false positives approach 0, the precision approaches 1?

Same question for recall:

r = true positives / (true positives + false negatives)

I am currently implementing a statistical test where I need to calculate these values, and sometimes it happens that the denominator is 0, and I am wondering which value to return for this case.

P.S.: Excuse the inappropriate tag, I wanted to use recall, precision and limit, but I cannot create new Tags yet.

Answer 1

Given a confusion matrix:

            predicted
            (+)   (-)
            ---------
       (+) | TP | FN |
actual      ---------
       (-) | FP | TN |
            ---------

we know that:

Precision = TP / (TP + FP)
Recall = TP / (TP + FN)

Lets consider the cases where the denominator is zero:

• TP+FN=0 : means that there were no positive cases in the input data
• TP+FP=0 : means that all instances were predicted as negative

Answer 2

Answer is Yes. The undefined edge cases occur when true positives (TP) are 0 since this is in the denominator of both P & R. In this case,

• Recall = 1 when FN=0, since 100% of the TP were discovered
• Precision = 1 when FP=0, since no there were no spurious results

This is a reformulation of @mbq's comment.

Answer 3

I am familiar with different terminology. What you call precision I would positive predictive value (PPV). And what you call recall I would call sensitivity (Sens). :

http://en.wikipedia.org/wiki/Receiver_operating_characteristic

In the case of sensitivity (recall), if the denominator is zero (as Amro points out), there are NO positive cases, so the classification is meaningless. (That does not stop either TP or FN being zero, which would result in a limiting sensitivity of 1 or 0. These points are respectively at the top right and bottom left hand corners of the ROC curve - TPR = 1 and TPR = 0.)

The limit of PPV is meaningful though. It is possible for the test cut-off to be set so high (or low) so that all cases are predicted as negative. This is at the origin of the ROC curve. The limiting value of the PPV just before the cutoff reaches the origin can be estimated by considering the final segment of the ROC curve just before the origin. (This may be better to model as ROC curves are notoriously noisy.)

For example if there are 100 actual positives and 100 actual negatives and the final segnemt of the ROC curve approaches from TPR = 0.08, FPR = 0.02, then the limiting PPV would be PPR ~ 0.08*100/(0.08*100 + 0.02*100) = 8/10 = 0.8 i.e 80% probability of being a true positive.

In practice each sample is represented by a segment on the ROC curve - horizontal for an actual negative and vertical for an actual positive. One could estimate the limiting PPV by the very last segment before the origin, but that would give an estimated limiting PPV of 1, 0 or 0.5, depending on whether the last sample was a true positive, a false positive (actual negative) or made of an equal TP and FP. A modelling approach would be better, perhaps assuming the data are binormal - a common assumption, eg: http://mdm.sagepub.com/content/8/3/197.short

Answer 4

That would depend on what you mean by "approach 0". If false positives and false negatives both approach zero at a faster rate than true positives, then yes to both questions. But otherwise, not necessarily.