Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals.

I was wondering if GBM models suffer in the case of imbalanced class sizes? I didn't find any references answering this question.

I tried to adjust the weights by assigning a weight of 1 to the uninfected individuals and a weight of 15 to the infected, but I obtained poor results.

share|improve this question
(side note) It would be helpful if you provided what GBM stands for and a link to the package. – Memming Aug 10 '15 at 12:58
Which loss function are you using for your gradient boosting model? When it comes to imbalanced classes, I've seen poor performance when I've used mean absolute error because it seems to favor the most common class. When I used mean squared error the performance improved substantially – Ryan Zotti Aug 12 '15 at 18:38
up vote 4 down vote accepted

In my experience, GBM does indeed suffer from imbalanced class sizes. I have had good success using SMOTE sampling, which creates synthetic data while oversampling the minority class. You can find it in the DMwR package.

share|improve this answer

I think your data is similar to Secom data on which I have worked in past and faced lot of difficulties. Following is what I have tried:

  • Different sampling techniques
  • Different classifiers like Random Forest, ANN, GBM, Ensemble methods, etc.

I've also tried 1-Class SVM which has given better results as compared to others like adaboost, Random Forest. You can try that as well.

And I can see you've asked this question 1 year back so if you've found the best way then kindly post it here so that I can get help from it to get better accuracy.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.