precision-recall curve is considered better than an ROC curve when testing a classifier on a dataset with a class imbalance.
This statement is wrong.
In general, statements like "X is better than Y" should be taken with a grain of salt. It usually depend on the use case, what is your target, etc. However, the statement above is more wrong than that. Let's take a look.
The PR curves plots the following parameters:
Precision = TP/(TP+FP)
Recall = TP/(TP+FN)
Notice how True Negatives (TN) are absent from the equation?
PR curves are useful when positive examples are rare. If your dataset is imbalanced with rare negatives, you should absolutely not use the PR curve.
As you noticed in your experiment, and as you have correctly reasoned, ROC curves are insensitive to class imbalance
. This means that if you balance your data (ie with resampling), the ROC curve does not change.
When the dataset contains only few positive examples, you have a new problem to care about: positive predictive value (= PPV = Precision). Specifically: given an observation is classified as positive, what is the probability that it is really a True Positive? The answer to this question can be surprisingly misleading when positive examples are rare.
PPV and NPV (the complement: given an observation is negative, what is the probability to be a true negative) are usually not an issue in balanced datasets, as they follow the usual sensitivity and specificity. PPV and NPV only become critical in imbalanced datasets, because these two measures are sensitive to class imbalance, unlike ROC curves. So ROC curves can obscure models with poor PPV and NPV, which can be an issue in the case of imbalance. PR curves will immediately highlight models with poor PPV and totally disregard NPV.
So in the end it is up to you to choose which tool to use. Don't use PR curves just because you have imbalance. Are positive examples rare? Are they rare specifically in your sample? Then stick with ROC curves. Are they rare in the general population? Then you should consider precision and look at the PR curve too. Are negative cases rare? Then stick with ROC curve.