8
$\begingroup$

For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics in the minority class (the "event" class) to better predict in the future, but my understanding, after reading through several papers and a few SAS blog posts today, is that oversampling when you already have the population isn't good practice, as you already have the entire population -- what more could you want?

In the case of logistic regression, oversampling wouldn't affect coefficients (outside of the slope intercept), so I don't see a reason to oversample in the case of that model. But what about for random forests or support vector machines? Would oversampling when I already have the entire population be a good or bad idea?

I guess my core question is: when shouldn't you oversample?

Improve this question
$\endgroup$
5
  • 8
    $\begingroup$ I've never got a clear answer on when it is a good practice: stats.stackexchange.com/questions/285231/… . There is the king paper, but the general oversampling methods don't seem to address the issues discussed there. Until otherwise, I'm inclined to think these methods are over-discussed nonsense. I would love to be corrected though. $\endgroup$
    – Matthew Drury
    Commented Aug 14, 2017 at 18:54
  • $\begingroup$ @MatthewDrury I take it you're not much a proponent of oversampling, then. My n is ~500,000, so it's not like I'm hurting for an absolute number of the positive class (with about 3,000 events). I'm thinking I'll just forget about oversampling in this case. $\endgroup$
    – boot-scootin
    Commented Aug 14, 2017 at 20:04
  • $\begingroup$ My take on this is that it depends on what you expect in the test sample. If you expect it to be imbalanced in a similar way then you can just use the appropriately threshold-ed probabilities, e.g. from random forest. If you want the trees to be more balanced (because you want the default threshold of 0.5 to be meaningful) then oversampling might help. You can also look at more sophisticated methods like the SMOTE. $\endgroup$
    – Krrr
    Commented Aug 15, 2017 at 8:35
  • 2
    $\begingroup$ What do you think oversampling can do that cannot be done with weights? Also see Frank Harrell's answer at stats.stackexchange.com/questions/199230/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Nov 6, 2020 at 16:38
  • $\begingroup$ If you already have the entire population, why do you need a model at all? You have some covariates; either the event happened for that combination of covariates or it did not. If you have multiple instances if the covariate combinations with different outcomes, you have that the outcome happened a certain proportion of the time and didn't happen some proportion of the time. $\endgroup$
    – Dave
    Commented Jan 14, 2022 at 20:22

1 Answer 1

Reset to default
3
$\begingroup$

If you have the entire population, there there is nothing to do. You know exactly what happened. If a subject had a certain combination of predictors (features) and experienced an outcome, then that was the outcome. If multiple subjects had the same combination of predictors and experienced different outcomes, you know the proportion in which they experienced the various outcomes.

You have the entire population. There is no need to do any modeling. Do not run logistic regressions. Do not run random forests. Do not run SVMs. You have the absolute, inarguable truth. It's like predicting yesterday's closing stock price. You don't predict it; you look up what it was, and that's what it was.

(In my opinion, even when we think we have the entire population, usually we are really interested in a data-generating process. If you find my response unsatisfactory, think hard about if you are interested in something beyond the subjects you observed.)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.