1
$\begingroup$

Let's say we have an unbalanced data set. We randomly sample an amount from our larger class so that we have a balanced data set. After tuning parameters/hyperparameters and determining which features to keep, do we train our final model on the full unbalanced data or the balanced data?

I would imagine we train it on the balanced dataset or else we would encounter the same issues that we were trying to avoid?

$\endgroup$
1
$\begingroup$

You need to train it on the balanced dataset, because 1. like you mentioned you will encounter the same problems, one of which is that the model will go towards predicting mostly the majority class and 2. your hyperparameters are trained on the balanced dataset

However, when you check the predictions on the test dataset, you do not need to balance it. It will tell you whether your "balancing" works

$\endgroup$
3
  • 1
    $\begingroup$ Perfect, that's what I've been doing. Thanks. $\endgroup$ – confused Apr 6 '20 at 9:18
  • $\begingroup$ So what do you suggest we do about all the unused data from the majority class? $\endgroup$ – user2974951 Apr 6 '20 at 9:42
  • $\begingroup$ If you have a large dataset, and those used in the training is enough to represent your majority class, it's not that useful. What you can try is to perform the sampling you described a few times, and use replicates of that to train your model for hyper-parameters $\endgroup$ – StupidWolf Apr 6 '20 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.