I am trying to train a Random Forest classifier on a binary classification problem but with a highly imbalanced dataset where the positive class is much smaller than the negative class . I have undersampled my dataset so that both classes have equal proportions and did 10 fold cross validation over this dataset.
However I am having bad results (averaged over the 10 folds):

  • recall : 46%
  • precision : 54%
  • accuracy : 53%

I was wondering if oversampling the positive class would yield better results. What would be the benefits of undersamling versus oversampling?

  • $\begingroup$ Did you have a look at the confusion matrix of your model's predictions? $\endgroup$
    – deemel
    Commented Jun 15, 2018 at 20:54
  • $\begingroup$ How many training samples do you have after undersampling? $\endgroup$
    – Jon Nordby
    Commented Jun 16, 2018 at 23:11

1 Answer 1


Undersampling at the data-collection phase can make sense, as you might be able to do more efficient data-collection. This is the topic of that King and Zeng paper discussed in the link.

However, once you have the data, unless you have computational constraints (cannot fit a huge dataset into memory, modeling is too slow even if you can, etc), undersampling makes no sense to me. I have my qualms with oversampling, too, but undersampling discards precious data. Since it turns out that class imbalance is rarely a problem for proper statistical methods, with those rare exceptions being outside the usual sense in many machine learning circles about why imbalance is a problem, discarding precious data to fix a non-problem seems like the worst idea of all.

Regarding oversampling, there probably is no need to do this. Proper statistical methods handle class imbalance just fine in most cases. However, at least you aren’t wasting data solving a non-problem when you oversample.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.