ROC AUC has $0.5$ as random performance. Does PR AUC have a similar notion?

Question

In considering ROC AUC, there is a sense in which $0.5$ is the performance of a random model. Conveniently, this is true, no matter the data or the prior probability of class membership; the ROC AUC of a random model is $0.5$ whether the binary classes are balanced or not.

Does PR AUC have a similar notion that is data-independent? (Since precision depends on the prior probability, I would imagine not.) If not, is there a notion that is a function of the prior probability?

EDIT

"ROC" means receiver-operator characteristic.

"PR" means precision-recall.

"AUC" means area under the curve.

Thus, "PR AUC" is the area under the curve that plots precision as a function of recall.

Answer 1

Firstly, you will want to have a look at precision-recall-gain curves, which enable comparison of classifier performance across datasets with different base rates. It's basically just a clever (and theoretically justified) nonlinear rescaling such that PRG curves cover [0,1]x[0,1] and are baseline-independent.

Snippet from Flach and Kull, NeurIPS 2015, see link below. Left: normal PR curve. Right: rescaled PRG curve.

Secondly, in PR(G) space, the baseline to beat is the always-positive classifier, not the random classifier. That model has precision=baseline and recall=1.

Thirdly, in their NeurIPS paper on PRG curves, Flach and Kull show that classifiers with the same $F_1$ score as the always-positive classifier lie on the (0,1)--(1,0) diagonal in PRG space. Any model operating point below that line thus has worse $F_1$ score than the always-positive classifier.

The only aspect that is still unclear to me (and they also don't write this explicitly in the paper) is whether that also implies a meaningful baseline of AUPRG=0.5 to beat, i.e., whether that baseline is also meaningful for the area under the curve, and not just pointwise. It seems to me that you could have AUPRG < 0.5, but as long as you have an operating point above the diagonal, the model could still be very useful if employed in that operating point? I think the AUROC=0.5 baseline relies crucially on the fact that any point below the diagonal can be "mirrored" by taking 1-the score instead of the actual score. It's not clear to me whether something like that can also be done in PR(G) space.

Lastly, if you really want to see what the random classifier is doing in PR(G) space, Flach and Kull also have an expression for its $F_1$ score as a function of the baseline. You may also be interested in this earlier paper on the relationship between PR and ROC curves.