Linked Questions

10 votes
0 answers

Baseline for Precision-Related Metrics [duplicate]

When working with ROC-AUC as a metric for binary classification, one often considers a value of 0.5 as a baseline from a random classifier (i.e. a data-blind classifier that randomly classifies test ...
Amelio Vazquez-Reina's user avatar
1 vote
0 answers

How to calculate the area under the precision-recall curve for the random classifier? [duplicate]

I know that the random classifier score in ROC AUC (Area under the curve) is always 0.5. My question is: how to calculate the Area under the precision-recall curve for the random classifier?
Jack Twain's user avatar
  • 8,201
256 votes
5 answers

ROC vs precision-and-recall curves

I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other. Do they always provide complementary insight about the performance of a ...
Amelio Vazquez-Reina's user avatar
18 votes
3 answers

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
M.M's user avatar
  • 524
21 votes
4 answers

Calculating AUPR in R [closed]

It is easy to find a package calculating area under ROC, but is there a package that calculates the area under precision-recall curve?
user avatar
13 votes
1 answer

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
RockTheStar's user avatar
  • 12.7k
10 votes
2 answers

What does it mean if the ROC AUC is high and the Average Precision is low?

I have a model that produces a high ROC AUC (0.90), but at the same time a low average precision (0.30). From what I've found, I think it might have to do something with imbalanced data (which the ...
Icyeval's user avatar
  • 515
8 votes
1 answer

PR AUC < 50% with ROC AUC > 90% - model good or bad?

I understand for highly imbalanced dataset - we need to look for precision-recall vs ROC AUC to better judge the model. My question is what is the range for PR AUC below which the model is bad? My ...
Vivek Singh's user avatar
11 votes
0 answers

Precision and recall of a random classifier

My understanding of precision and recall tells me that there is a tradeoff between these two measures: you can improve one at the cost of the other. However, when I think of a random classifier (on a ...
GermanK's user avatar
  • 211
4 votes
1 answer

Area Under the Precision Recall curve -similar interpretation to AUROC?

I am trying to interpret the AUCPR. Say I have the following Precision-Recall curve. Firstly: It ends at 0.38 on the y-axis because this particular plot has ...
user113156's user avatar
1 vote
1 answer

What does a "flat region" of precision recall curve imply?

I am evaluating ML models (GBDTs) on various test sets using Precision-recall curve, and my goal is: within some precision range, get as high recall as possible. The precision-recall curves on most of ...
Zero Liu's user avatar
3 votes
1 answer

ROC AUC has $0.5$ as random performance. Does PR AUC have a similar notion?

In considering ROC AUC, there is a sense in which $0.5$ is the performance of a random model. Conveniently, this is true, no matter the data or the prior probability of class membership; the ROC AUC ...
Dave's user avatar
  • 58.5k
1 vote
1 answer

What is the expected value of AUROCC for random predictions?

I was having a debate with co-workers today about the dependence of AUC on class imbalance, ie, the proportion of positive/negative instances in the response variable. It was suggested that when ...
alan ocallaghan's user avatar
2 votes
1 answer

What is "better-than-random" precision in clustering?

In Section 6.3.1 of the paper "No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems", it is mentioned that the algorithm proposed by the paper has ...
Curious's user avatar
  • 411
1 vote
0 answers

F1 Score vs PR Curve

If I understood correctly, PR Curve it's just the mean of F1 score computed multiple times with different thresholds. In the task of outlier detection those are two suggested metrics given the fact ...
Loris's user avatar
  • 23