12 questions linked to/from ROC vs precision-and-recall curves
31k views

### Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
11k views

### ROC vs Precision-recall curves on imbalanced dataset

I just finished reading this discussion. They argue that PR AUC is better than ROC AUC on imbalanced dataset. For example, we have 10 samples in test dataset. 9 samples are positive and 1 is ...
10k views

### What is “baseline” in precision recall curve

I'm trying to understand precision recall curve, I understand what precision and recall are but the thing I don't understand is the "baseline" value. I was reading this link https://classeval....
8k views

### “Good” classifier destroyed my Precision-Recall curve. What happened?

I'm working with imbalanced data, where there are about 40 class=0 cases for every class=1. I can reasonably discriminate between the classes using individual features, and training a naive Bayes and ...
5k views

### Choosing between loss functions for binary classification

I work in a problem domain where people often report ROC-AUC or AveP (average precision). However, I recently found papers that optimize Log Loss instead, while yet others report Hinge Loss. While I ...
7k views

### How can I calculate the false positive rate for an object detection algorithm, where I can have multiple objects per image?

I have an object detection algorithm for which I would like to plot an ROC curve. For this, I need the values of the fall-out corresponding to values of recall. The false positive rate, or fall-out, ...
2k views

### outlier detection: area under precision recall curve

I would like to compare outlier detection algorithms. I am not sure if area under roc or under precision recall curve is the measure to use. A quick test in matlab gives me strange results. I try to ...
248 views

### Imbalanced data-set : rare class v.s. rare events

I am currently working on an imbalanced data-set (1% of 1). However I am a bit concerned by the underlying model. I treated the problem as a classification problem, making some hypotheses on the ...
142 views

### performance measure suited for imbalanced classes and robust towards changing class ratios

I am looking for the best performance measure. My use case: I want to find out which dataset can be modelled best with binary classification. The datasets have an active minority class I am ...
49 views

### precision recall AUC or ROC AUC question

I'm working on a project where the 'in the wild' prevalence is a significant imbalance (e.g. minority 4%). However, I was able to collect data in a balanced manner, i.e. 4,000 samples of minority and ...