I have an imbalanced dataset (90% class 0 10% class 1), should I first split it in training and test set, then balance my training set (my test set would still be imbalanced), or could I randomly downsample the majority class in my dataset then split in training and test set (my test set will not contain any observations used in the training set because I down sampled)? I am getting very different results for each approach.


1 Answer 1


I would say neither of the options you suggested: use all the data you have and try to ensure the distributions of training and test set match. This will give you the most realistic assessment of the model's performance. If your classification algorithm has trouble dealing with imbalanced data, use something else.

So in short: split in training and testing, keeping both of them imbalanced. Don't throw data away by subsampling.

  • $\begingroup$ Hi Marc, would appreciate any insight you could give here. $\endgroup$
    – Spacey
    Jan 30, 2017 at 5:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.