Is Undersampling or oversampling for unbalanced classification data?

Question

I am trying to train a Random Forest classifier on a binary classification problem but with a highly imbalanced dataset where the positive class is much smaller than the negative class . I have undersampled my dataset so that both classes have equal proportions and did 10 fold cross validation over this dataset.
However I am having bad results (averaged over the 10 folds):

• recall : 46%
• precision : 54%
• accuracy : 53%

I was wondering if oversampling the positive class would yield better results. What would be the benefits of undersamling versus oversampling?

Answer 1

Undersampling at the data-collection phase can make sense, as you might be able to do more efficient data-collection. This is the topic of that King and Zeng paper discussed in the link.

However, once you have the data, unless you have computational constraints (cannot fit a huge dataset into memory, modeling is too slow even if you can, etc), undersampling makes no sense to me. I have my qualms with oversampling, too, but undersampling discards precious data. Since it turns out that class imbalance is rarely a problem for proper statistical methods, with those rare exceptions being outside the usual sense in many machine learning circles about why imbalance is a problem, discarding precious data to fix a non-problem seems like the worst idea of all.

Regarding oversampling, there probably is no need to do this. Proper statistical methods handle class imbalance just fine in most cases. However, at least you aren’t wasting data solving a non-problem when you oversample.