Imbalanced data classification using boosting algorithms

Question

I am working on a binary data classification problem. The dataset is imbalanced, it consists of 92% 'false' labels and 8% 'true' labels. The number of features is 18 and I have a small number of 650 data points. I want to use boosting algorithms in MATLAB like GentleBoost to solve this problem. I assign Uniform for prior as follows:

ada = fitensemble(Xtrain,Ytrain,'GentleBoost',10,'Tree','LearnRate',0.1, 'prior', 'uniform')

but the performance is consistently poor. How should I set the parameters? Is it necessary to set a cost? How can I do this? Is there any classifier that perform better than this?

Answer 1

If you have R2012b or later, use the RUSBoost algorithm. It is recommended for imbalanced datasets.

If you go with GentleBoost, you need to optimize the tree complexity and the number of trees in the ensemble. (You could also play with the learning rate.) Both parameters are likely far off their optimal values in your code.

First, fitensemble for GentleBoost by default produces decision stumps (trees with two leaves). Since the minority class is only 8% of the data, stumps are not sensitive to observations of the minority class. I often set the minimal leaf size to one half of the size of the minority class. The optimal setting for the leaf size may not be exactly that but should be in that ballpark. Do:

tmp = ClassificationTree.template('minleaf',some_number);
ens = fitensemble(Xtrain,Ytrain,'GentleBoost',Ntrees,tmp,'prior','uniform')

Second, 10 trees are most usually not enough. Inspect the ensemble accuracy by cross-validation or using an independent test set to decide how many trees are needed. Typically, a few hundred should be used for boosting.

Also, after you train the ensemble, don't just look at the classification error. Use the perfcurve function to compute a performance curve and find the optimal threshold on the classification score. It is up to you to define what "optimal" means. You can assign, for instance, different misclassification costs to the two classes and find the threshold minimizing the expected cost. .....