0
$\begingroup$

I have unbalanced data so I want to oversample obs from the minority class and then apply Logistic regression to the training set. After that, I would like to perform cross-validation. My question is: When should I separate the data into training and test? After oversampling? Any help will be appreciated.

$\endgroup$
8

1 Answer 1

1
$\begingroup$

As some links I posted discuss, class imbalance usually is not a problem. Therefore, attempts to solve a non-problem are somewhere between superfluous and damaging. Consequently, it is difficult to suggest where you would apply such a technique.

However, I have applied poor practices in regression models specifically to show how they fail, so there is a place for knowing where in the workflow such a practice would occur.

Any messing with the data would come after you have set aside out-of-sample data, since out-of-sample data are there to mimic the real-world application of your model on data that might not even exist (such as Siri or Alexa being expected to be able to do speech recognition on words spoken by people who have yet to speak their first words). This is what Dikran Marsupial means in the comment about doing resampling in the training folds of a cross-validation but not the test partition.

I don’t know where exactly that is in your particular package. It might be that you will have to code the splitting and cross-validation yourself (probably not, but maybe). However, it would be cheating to do ROSE on data that are supposed to be hidden.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.