For a binary classification, let's say we have samples from two classes, namely class A and class B. Is it a must to have the same number of samples from each class? i.e. Is it a bad practice to have a dataset of 30% A & 70% B for a machine learning problem?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
No, it certainly isn't. Your training sample should ideally be representative of the population you later want to apply the trained model to. If you, e.g., oversample the minority class, you pretend that the base population is different from what it truly is.
If your classes are strongly unbalanced, a naively trained classifier may classify everything as the majority class. This is not a problem of the training set, but one of inappropriate in-sample quality measures like sensitivity or specificity. Instead, have your classifier output probabilities and assess these using proper scoring-rules.
More information and links to explanations by Frank Harrell here: Rare Events Logistic Regression
answered Nov 8, 2017 at 11:43