What are the best practices for fitting a binomial classification model when the classes are very imbalanced?
For example, 99.9% 1's and 0.1% 0's.
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Resampling methods that attempt to produce balanced data sets are popular. Concretely SMOTE, which creates new samples that follow the distribution of present samples. Resampling methods may also be used together with ensemble methods (like bootstrapping) in order to increase accuracy.
Since it is hard to fit many models under that setting properly, one option is to do novelty detection: learn about the distribution of 1's and find a threshold that provides good accuracy on the test set. For example, you may use SVM's for novelty detection.
$\begingroup$
$\endgroup$
Often problem is not the class imbalance, rather it is the number of cases in the rare class.
If you do class balancing, it is important to know that probabilities which model produces are not population level probabilities. They are conditional on the sampling scheme used.
