2
$\begingroup$

I have a unbalanced data set, that for the purpose of training I wish to balance. What is a better practice? First balancing the train data and then feature scaling/normalizing or using the mean and standard deviation of the skewed set?

Thanks for your help! =)

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

Are you talking about a predictive model? I will assume yes. Also assume you are talking about balancing the target variable for rare class classification tasks. My thought would be to standardize (normalizing is typically using the min and max values not mean and standard deviation) the data first and then over-sample if that is what you are thinking in terms of balancing. I say this because you will want to use that same mean/std dev of the original set when you standardize new data so that it mirrors the training set that was used. The suggestion above not to balance is not true, it really depends on how rare the rare class is and the technique. Often, machine learning algorithms work better with closer to balance data (either explicitly or through use of weights).

Improve this answer
$\endgroup$
11
  • $\begingroup$ "The suggestion above not to balance is not true, it really depends on how rare the rare class is and the technique. Often, machine learning algorithms work better with closer to balance data (either explicitly or through use of weights)." — Why do you say so? I don't see why a model would be improved by distorting the training data in this way. Weighting of data points makes sense if you know some points are more important or more informative than others, but not merely as an attempt to compensate for imbalanced data. $\endgroup$
    – Kodiologist
    Commented Jan 6, 2017 at 17:30
  • $\begingroup$ This is a pretty well known issue and you can find lots of things in Google. For example, a couple quick links of the practical variety. svds.com/learning-imbalanced-classes analyticsvidhya.com/blog/2016/03/… Empirically, we find better performance often (read: not always - there is no free lunch) in closer to balanced classes. This doesnt mean 50/50 when the rare class is very rare - that excludes too much information from the majority class. It means oversampling (for example) to achieve say 1% rare class $\endgroup$
    – B_Miner
    Commented Jan 6, 2017 at 18:18
  • 1
    $\begingroup$ @B_Miner From your links, I can't see how these things could improve a classifier's performance except when you're using a metric under which one kind of error (false positive vs. false negative) is more important than the other. And when that's case, that suffices to justify weighting even when the data is balanced. So imbalance seems a bit of a red herring. $\endgroup$
    – Kodiologist
    Commented Jan 7, 2017 at 1:15
  • 1
    $\begingroup$ @B_Minder "Not sure your point." — My point is that an asymmetric loss function is necessary and sufficient to justify reweighting (or, since you say this isn't always possible, oversampling or undersampling). If you're using a symmetric measure, like accuracy, then you're not going to benefit from such things no matter how imbalanced the data is to start with. $\endgroup$
    – Kodiologist
    Commented Jan 9, 2017 at 15:48
  • 1
    $\begingroup$ Over/under sampling is a symptom of not understanding limits of classification and improper accuracy scoring rules. See fharrell.com/post/classification $\endgroup$
    – Frank Harrell
    Commented Dec 11, 2018 at 20:15
-2
$\begingroup$

I have a unbalanced data set, that for the purpose of training I wish to balance.

By which you mean, throw out or duplicate observations? This is unwise. It will only make your model worse.

So, just normalize and forget about balancing.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Over- and under-sampling are known to improve machine learning models in many classification tasks. See e.g. arxiv.org/pdf/1608.06048.pdf $\endgroup$
    – Denwid
    Commented Dec 11, 2018 at 20:46
  • $\begingroup$ @Denwid See my comments on the other answer: "an asymmetric loss function is necessary and sufficient to justify reweighting". I admit that duplicating or removing observations is better than nothing when your model doesn't admit weighting… but if you have an asymmetric loss function, you should probably be using a model that admits weighting. $\endgroup$
    – Kodiologist
    Commented Dec 11, 2018 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.