6
$\begingroup$

My understanding of precision and recall tells me that there is a tradeoff between these two measures: you can improve one at the cost of the other.

However, when I think of a random classifier (on a binary task) that outputs class $1$ with probability $p$, I don't observe any tradeoff.

$Prec= \frac{TP}{TP+FP} = \frac{p \cdot P}{p \cdot P + p \cdot N} = \frac{P}{P + N}$

$Rec = \frac{TP}{TP+FN} = \frac{p \cdot P}{p \cdot P + (1-p) \cdot P} = p$

Hence, precision is constant (it is given by the prevalence of the positive class), and recall is a high as I want it to be.

Is there anything wrong here?

$\endgroup$
  • $\begingroup$ Your reasoning is correct (asymptotically) under the assumption of an entirely random classifier. I'm not sure how this conclusion is useful, though. $\endgroup$ – Marc Claesen Mar 10 '14 at 17:43
  • $\begingroup$ Wonder what you mean by "asymptotically correct" :) $\endgroup$ – GermanK Mar 11 '14 at 9:13
  • $\begingroup$ Well, I was thinking about how my classifier compares to a random one (I know there are many questions on the issue) and I was expecting some kind of balance between precision and recall, as the one that exists on a ROC curve where when you increase TPR, you automatically increase the FPR (still, for a random classifier). I was then surprised to find out that there is not such a tradeoff in this case: the best random classifier to compare with, when using P-R to measure it, is just the one that classifies as positive with $p=1$, since precision is fixed and $Recall=p$ $\endgroup$ – GermanK Mar 11 '14 at 9:21
  • 1
    $\begingroup$ Yes, you are right. That part of my comment was unnecessary. A useful application of this observation is that it allows one to compute a baseline PR curve for a random classifier, similar to the $45^\circ$ line in ROC analysis. Basically, a random classifier which outputs positive with probability $p$ has area under the PR curve equal to $p$. $\endgroup$ – Marc Claesen Mar 11 '14 at 9:36
  • 1
    $\begingroup$ Exactly, and that is, surprisingly for me, a flat line! And, since you don't gain anything with lowering recall, all the baselines in that curve are trivially dominated by the classifier where $p=1$. $\endgroup$ – GermanK Mar 11 '14 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.