Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Precision is defined as:
p = true positives / (true positives + false positives)
What is the value of precision if (true positives + false positives) = 0? Is it just undefined?
Same question for recall:
r = true positives / (true positives + false negatives)
In this case, what is the value of recall if (true positives + false negatives) = 0?
P.S. This question is very similar to the question What are correct values for precision and recall in edge cases?.
The answers to the linked earlier question apply here too.
If (true positives + false negatives) = 0 then no positive cases in the input data, so any analysis of this case has no information, and so no conclusion about how positive cases are handled. You want N/A or something similar as the ratio result, avoiding a division by zero error
If (true positives + false positives) = 0 then all cases have been predicted to be negative: this is one end of the ROC curve. Again, you want to recognise and report this possibility while avoiding a division by zero error.
An interesting answer is offered here: https://github.com/dice-group/gerbil/wiki/Precision,-Recall-and-F1-measure
The authors of the module output different scores for precision and recall depending on whether true positives, false positives and false negatives are all 0. If they are, the outcome is ostensibly a good one.
In some rare cases, the calculation of Precision or Recall can cause a division by 0. Regarding the precision, this can happen if there are no results inside the answer of an annotator and, thus, the true as well as the false positives are 0. For these special cases, we have defined that if the true positives, false positives and false negatives are all 0, the precision, recall and F1-measure are 1. This might occur in cases in which the gold standard contains a document without any annotations and the annotator (correctly) returns no annotations. If true positives are 0 and one of the two other counters is larger than 0, the precision, recall and F1-measure are 0.
I'm not sure if this type of scoring would be useful in other situations outside of their special case, but it's worth some thought.
When evaluating a classifier at high thresholds, the precision might (often actually) not be 1 when recall is 0. It's usually N/A! I think there is something wrong about how people plot the P/R curve. Avoiding N/A samples is a bias in the sense that you avoid singularity samples. I computed the average precision wrt to the average recall ignoring N/A samples and I never got a classifier starting at 1 for 0 recall for a shallow neural net in object detection. This was also true for curves computed with the tp,fp,fn numbers. It's quite easy to verify by paper and pencil with a single image. For example: I have a classifier that outputs for a single image: preds=[.7 .6 .5 .1 .05] truth=[n y n n y ] By computing the confusion matrices with the various thresholds we have: tp=[2 1 1 1 0 0],fn=[0 1 1 1 2 2],fp=[3 3 2 1 1 0]. the recall rec=[1 .5 .5 .5 0 0], and the precision=[.4 .25 1/3 .5 0 NaN]. I don't see how it would make sense to replace a NaN or the precision(@recall==0) with 1. 1 should be an upper bound, not a value we replace precision(@recall==0) with.
