Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a data set which is highly imbalanced and I have used the SMOTE algorithm (using the R package DMwR) to balance the binary class in the data set. I have been using the R Ada package to then train an Ada Boost model on this data set to predict the binary class, with very good results.

In the same data set, I have another class variable which has multiple values (6 in total). In this case I realise that I can't use the AdaBoost algorithm as implemented in the ada package as it only deals with the binary case.

I therefore have 2 problems:

  1. I'd like to use the SMOTE algorithm on second class variable but this also only works with binary classes. Is there an algorithm or package I can use in R to "rebalance" a data set based on a class with multiple values in a similar way to SMOTE?

  2. I'd like to use a classifier to predict the multiple class variable. I have tried using the one-vs-all approach with AdaBoost but I cannot get this to work well (my approach is below). Boosting seems to work well with this data set. Are there any other boosting algorithms or other approaches I could use in R that handle classes with multiple values. I have tried using Random Forest but one of my nominal inputs has too many discrete values to use it.

Approach for AdaBoost one-vs-all

  • Build a vector with a binary variable for each discrete class value
  • Train one AdaBoost model against each binary class vector
  • Generate probability prediction for each AdaBoost model
  • Select the class with the highest probability

Many thanks

share|improve this question
    
From my neural network experience, could you make the 6 values into binary values such as 1000000, 0100000, 0010000, 0001000, etc and then use adaboost. just a thought. –  user14047 Sep 13 '12 at 0:08
    
Has any solution to this been found? –  Oeufcoque Penteano Oct 13 '12 at 3:06

1 Answer 1

You can use maboost package in R. It implements mutliclass boosting. Its multiclass boosting is in a sense the generalization of adaboost.MM. It directly solves the multiclass boosting without reduce it to binary classification problems and perhaps appropriate for your application.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.