1
$\begingroup$

I want to run a classification tree using rpart but the variable that I want to predict has a lot of class imbalance:

Behaviour:
a     b     c     d     e
35    100   32    405   34301

I have downsampled the data for Random Forest (RF) but when I split into 2/3 training and 1/3 test data for the classification tree and then downsample to the minority class, I lose alot of data. I've tried using SMOTE but it seems that only works when there are binary classes. It oversamples "c" and downsamples everything else so that "a" and "b" are around zero in order to bring "e" down to a suitable size. Cost sensitive learning doesn't look to be an option as I can't implement it properly in RF and I want to compare the two. I'd really appreciate any suggestions.

$\endgroup$

migrated from stackoverflow.com Aug 7 '16 at 10:37

This question came from our site for professional and enthusiast programmers.

1
$\begingroup$

Lets take the issue in hand step by step. First training and test data creation. For this you need to do stratified sampling just to get proper representation of both behaviors in both parts. You can do so in data.table by:

train_ind <- data[, sample(.I, round(0.66*.N), FALSE),by="behavior"]$V1

Now to treat imbalance you have many ways in RF:

  1. Put higher penalty of misclassification for rarer classes.
  2. Increase penalty even more with sample weight.

Due to presence of first two options in neat way I like Python scikit-learn RF more. However if you want to do it in R then I would suggest you to use caret and use appropriate RF version where these options are available (choose from http://topepo.github.io/caret/Random_Forest.html).

I have dealt with even more skewed class in my project and based on my experience I like xgboost (gbm is there but its slow) much more to deal with such problem.

$\endgroup$
  • $\begingroup$ All great suggestions. I'd just add that, from my own experience with highly unbalanced datasets, using a proper metric to evaluate performance is essential. Have you tried using the precision-recall-gain? $\endgroup$ – darXider Jul 28 '17 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.