For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics in the minority class (the "event" class) to better predict in the future, but my understanding, after reading through several papers and a few SAS blog posts today, is that oversampling when you already have the population isn't good practice, as you already have the entire population -- what more could you want?

In the case of logistic regression, oversampling wouldn't affect coefficients (outside of the slope intercept), so I don't see a reason to oversample in the case of that model. But what about for random forests or support vector machines? Would oversampling when I already have the entire population be a good or bad idea?

I guess my core question is: when shouldn't you oversample?

  • 4
    $\begingroup$ I've never got a clear answer on when it is a good practice: stats.stackexchange.com/questions/285231/… . There is the king paper, but the general oversampling methods don't seem to address the issues discussed there. Until otherwise, I'm inclined to think these methods are over-discussed nonsense. I would love to be corrected though. $\endgroup$ – Matthew Drury Aug 14 '17 at 18:54
  • $\begingroup$ @MatthewDrury I take it you're not much a proponent of oversampling, then. My n is ~500,000, so it's not like I'm hurting for an absolute number of the positive class (with about 3,000 events). I'm thinking I'll just forget about oversampling in this case. $\endgroup$ – blacksite Aug 14 '17 at 20:04
  • $\begingroup$ My take on this is that it depends on what you expect in the test sample. If you expect it to be imbalanced in a similar way then you can just use the appropriately threshold-ed probabilities, e.g. from random forest. If you want the trees to be more balanced (because you want the default threshold of 0.5 to be meaningful) then oversampling might help. You can also look at more sophisticated methods like the SMOTE. $\endgroup$ – DataD'oh Aug 15 '17 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.