11
$\begingroup$

My understanding of precision and recall tells me that there is a tradeoff between these two measures: you can improve one at the cost of the other.

However, when I think of a random classifier (on a binary task) that outputs class $1$ with probability $p$, I don't observe any tradeoff.

$Prec= \frac{TP}{TP+FP} = \frac{p \cdot P}{p \cdot P + p \cdot N} = \frac{P}{P + N}$

$Rec = \frac{TP}{TP+FN} = \frac{p \cdot P}{p \cdot P + (1-p) \cdot P} = p$

Hence, precision is constant (it is given by the prevalence of the positive class), and recall is a high as I want it to be.

Is there anything wrong here?

Improve this question
$\endgroup$
7
  • $\begingroup$ Your reasoning is correct (asymptotically) under the assumption of an entirely random classifier. I'm not sure how this conclusion is useful, though. $\endgroup$
    – Marc Claesen
    Mar 10, 2014 at 17:43
  • $\begingroup$ Wonder what you mean by "asymptotically correct" :) $\endgroup$
    – GermanK
    Mar 11, 2014 at 9:13
  • 1
    $\begingroup$ Well, I was thinking about how my classifier compares to a random one (I know there are many questions on the issue) and I was expecting some kind of balance between precision and recall, as the one that exists on a ROC curve where when you increase TPR, you automatically increase the FPR (still, for a random classifier). I was then surprised to find out that there is not such a tradeoff in this case: the best random classifier to compare with, when using P-R to measure it, is just the one that classifies as positive with $p=1$, since precision is fixed and $Recall=p$ $\endgroup$
    – GermanK
    Mar 11, 2014 at 9:21
  • 1
    $\begingroup$ Yes, you are right. That part of my comment was unnecessary. A useful application of this observation is that it allows one to compute a baseline PR curve for a random classifier, similar to the $45^\circ$ line in ROC analysis. Basically, a random classifier which outputs positive with probability $p$ has area under the PR curve equal to $p$. $\endgroup$
    – Marc Claesen
    Mar 11, 2014 at 9:36
  • 1
    $\begingroup$ Exactly, and that is, surprisingly for me, a flat line! And, since you don't gain anything with lowering recall, all the baselines in that curve are trivially dominated by the classifier where $p=1$. $\endgroup$
    – GermanK
    Mar 11, 2014 at 9:42

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.