I have some imbalanced data for binary classification, which I have preprocessed in 2 different ways. That led to having a different number of observations and pos/neg ratio. Then I trained the same model (same parameters, except class weights), let's say Logistic Regression, and have achieved the next results:
| AUPRC | Baseline | delta |
|---|---|---|
| 0.34 | 0.22 | 0.12 |
| 0.43 | 0.27 | 0.16 |
delta == AUPRC - Baseline
AUPRC is an area under Precision-Recall curve.
It’s a bit trickier to interpret AUPRC than it is to interpret AUROC (the area under the receiver operating characteristic). That’s because the baseline for AUROC is always going to be 0.5 — a random classifier, or a coin toss, will get you an AUROC of 0.5. But with AUPRC, the baseline is equal to the fraction of positives, where the fraction of positives is calculated as (# positive examples / total # examples).
- Is it legitimate to say, looking at deltas, that the second case is better than the first one? -- 0.12 vs 0.16
- Can I say that the second case is 33% more accurate than the first one? --
0.16 / 0.12 - 1 ~= 0.33
The main idea is to show that more accurate data preprocessing leads to better target metric results (more "accurate" or "precise").