0
$\begingroup$

As I understood till now, ROC curves are not a good presentation of unbalanced datasets and PR curves are preferred because ROC curves are not sensitive to false positives. If we now use resample strategies like over or undersampling however, would ROC curves be a good representation again? Is there even a need for over and under sampling if the minority class still contains around 10,000 cases for training the model (in case of probability prediction)?

$\endgroup$
1
$\begingroup$

Define what you mean by 'good' in that context. Often, ROC curves are used in situations of class imbalance specifically because they are not susceptible to class imbalance but rather indicate the general ability of a model to separate classes, i.e. their ability to produce a meaningful score.

As to whether or not one should re-sample the training set in case of class imbalance - I'm not sure. Some people showed that some algorithms perform worse on unbalanced data, e.g. the C4.5 trees, while others argue that the imbalance problem is in fact just a data sparsity problem and that most problems people commonly observe in context of class imbalance are in fact just due to an insufficient number of sample of the minority class (see Provost et al).

Personally, I'm not convinced that class imbalance poses an actual problem and I think the 'problem' of class imbalance arose simply due to accuracy being a common measure in machine learning (which it really shouldn't be, see this answer).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.