1
$\begingroup$

I have built a classification model to recognise a class and I have evaluated it on several datasets. The problem is that some of these datasets do not have any true instance of the class in question, and so, the model should ideally recognise zero instances.

If the model has 0 false positives on such a dataset, I may claim that the precision of this model on this dataset is 100%. Conceptually, a classification model that recognises 0 instances out of 0 with no FP is a precise model. If, however, the model has only one false positive, the precision will drop down to 0 although a model with only one FP may generally be assumed to have a good precision.

With regards to the recall, since the TP and FN will always be 0, may I claim that the recall on such datasets should always be 100%?

If not, what is the appropriate way to calculate precision/recall in such cases? I am, by the way, aware of the existence of other accuracy evaluation measures but I need to calculate these two (i.e. precision and recall).

Improve this question
$\endgroup$
4
  • $\begingroup$ Precision and recall, by definition, require positives. There is no way around it without changing their definition. $\endgroup$
    – Marc Claesen
    Commented Aug 13, 2014 at 14:09
  • $\begingroup$ In my view, I think, even without any positives, recognising 0 instances out of 0 with FP=0 is a good precision, isn't it? Precision may be defined to equal 1 if positives=0 and fp=0 and tp/(tp+fp) otherwise (I have just changed its definiton, haven't I?!). I agree with you, their definition may need to be extended or changed to account for such a possibility. $\endgroup$
    – Sultan Abraham
    Commented Aug 13, 2014 at 14:41
  • 2
    $\begingroup$ I disagree. It just makes no sense to talk about precision and recall without positives. The reason you want to mention these is to assess the quality of a model. These measures can't give any useful information, since you can't distinguish a good model from one that always predicts negative in your case. $\endgroup$
    – Marc Claesen
    Commented Aug 13, 2014 at 14:57
  • $\begingroup$ That is a good point. Could you please add your comments as an answer so I can accept it? $\endgroup$
    – Sultan Abraham
    Commented Aug 13, 2014 at 15:04

1 Answer 1

Reset to default
4
$\begingroup$

In my opinion it makes no sense to talk about precision and recall without positives. The reason you want to mention these is to assess the quality of a model. These measures can't give any useful information, since you can't distinguish a good model from a trivial one that always predicts negative in your case.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.