I’m working on a classification problem where dataset is extremely imbalanced ( roughly 13000 "zero" and 100 "one" responses).

As the first step, I trained a Logistic Regression and changing the cutoff probability, managed to predict most of the “one” responses correctly, but a reasonable number of “zero” responses were incorrectly classified as “one”.

So I would like to know that, what are the good algorithm which can properly handle imbalance datasets?


P.S. I’m looking at algorithms which are available in scikit-learn or as a R package.


I recommend re-sampling techniques to balance the training dataset. They can be divided in four categories: undersampling the majority class, oversampling the minority class, combining over and under sampling, and creating an ensemble of balanced datasets.

The above methods and more are implemented in the imbalanced-learn library in Python that interfaces with scikit-learn. I recommend trying an combined method such as SMOTE + Tomek links to see if classification accuracy improves on a balanced dataset.

See ipython notebook for an example.

  • $\begingroup$ Why do you reccomend these? What problem do they solve? $\endgroup$ – Matthew Drury Aug 1 '17 at 0:12
  • $\begingroup$ most classification algorithms will perform optimally when the number of samples in each class is roughly the same, one can use re-sampling to arrive at a more accurate decision boundary. E.g. SMOTE generates synthetic samples using KNN while Tomek links remove unwanted overlap between classes. See ieeexplore.ieee.org/document/5128907 for a review. $\endgroup$ – Vadim Smolyakov Aug 1 '17 at 17:44
  • $\begingroup$ I don't believe that is true, and it not the consensus in the questions on this site: stats.stackexchange.com/questions/283170/… $\endgroup$ – Matthew Drury Aug 1 '17 at 20:06
  • $\begingroup$ yeah, i found little discussion on which algorithms are affected the most by the imbalanced datasets. i can imagine imbalanced data could be a problem for a simple online learning algorithm like perceptron where the order of points matters in updating the classification boundary, in the case of perceptron the decision boundary will look different if the data classes were roughly balanced compared to seeing the majority class labels; also in decision trees, for imbalanced data, it's easy to achieve high accuracy just by predicting the majority class label all the time. $\endgroup$ – Vadim Smolyakov Aug 1 '17 at 21:08
  • $\begingroup$ @MatthewDrury While the imbalance may not be a problem theoretically wise it makes inference harder technically/numerically. For one, it distorts "conventional" classification performance metrics like AUROC, accuracy etc. So one needs to come up with something "exotic" like F1, Jaccard index, Kappa or the like. Also, during the model selection process of cross-validation or bootstrapping it may happen so that some steps would miss the minority class entirely if the imbalance is heavy enough. $\endgroup$ – ayorgo Sep 21 '19 at 17:48

I would use stratified sampling methods (e.g. http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation-iterators-with-stratification-based-on-class-labels) Sorry for not being able to give a precise strategy nor an explanation what exactly these algorithms do in the background, but I hope this helps nonetheless.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.