I have some imbalanced data for binary classification, which I have preprocessed in 2 different ways. That led to having a different number of observations and pos/neg ratio. Then I trained the same model (same parameters, except class weights), let's say Logistic Regression, and have achieved the next results:

1. Is it legitimate to say, looking at delta (ROC AUPRC - baseline), that the second case is better than the first one?
2. Can I say that the second case is 25% more accurate than the first one?

The main idea is to show that more accurate data preprocessing leads to better accuracy.

I have some imbalanced data for binary classification, which I have preprocessed in 2 different ways. That led to having a different number of observations and pos/neg ratio. Then I trained the same model (same parameters, except class weights), let's say Logistic Regression, and have achieved the next results:

delta == AUPRC - Baseline

AUPRC is an area under Precision-Recall curve.

It’s a bit trickier to interpret AUPRC than it is to interpret AUROC (the area under the receiver operating characteristic). That’s because the baseline for AUROC is always going to be 0.5 — a random classifier, or a coin toss, will get you an AUROC of 0.5. But with AUPRC, the baseline is equal to the fraction of positives, where the fraction of positives is calculated as (# positive examples / total # examples).

1. Is it legitimate to say, looking at deltas, that the second case is better than the first one? -- 0.12 vs 0.16
2. Can I say that the second case is 33% more accurate than the first one? -- 0.16 / 0.12 - 1 ~= 0.33

The main idea is to show that more accurate data preprocessing leads to better target metric results (more "accurate" or "precise").