2
$\begingroup$

Can someone point me to some resource (textbook,paper,blog,..) that clearly explains when we should NOT balance data for classification/regression?

I found https://matloff.wordpress.com/2015/09/29/unbalanced-data-is-a-problem-no-balanced-data-is-worse/ but wonder if there are other cases and/or more complete answer for the question.

$\endgroup$
-2
$\begingroup$

Balancing data is used to account for sample bias - to make sample data more closely represents the actual population. If there isn't sample bias, it's probably better not to balance out the data. Doing so could very well negatively impact the accuracy of your algorithm.

Suppose we know 10% of transactions in some database are fraudulent, but only 1% were labeled as such, maybe to save face in the public eye. A well-built algorithm trained on this misleading data would be pretty bad at detecting new fraudulent cases. This is a situation where we might balance the classes.

If it really was true that only 1% of the transactions were fraudulent, balancing could be a bad idea, as it would introduce bias into our data - we're no longer training our algorithm on 'reality'.

You shouldn't balance your classes unless it improves your algorithm's accuracy on unseen data.
If the unbalanced dataset accurately represents the native population, try it 'as is' while being careful with your train/validation/test splits, and using evaluation metrics appropriate for skewed data like precision or recall. Leverage this information for ideas on how to improve the algorithm before deciding to balance things out.

Remember, 'premature optimization is the root of all evil'.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.