# The Best Approach for the Classification of the imbalanced classes

I have data that I am going to classify to 3 classes, but one class has a few samples less than 5% of the total samples. What is the best approach to classify these imbalanced classes? I mean is there a machine learning technique that fits this case or is there any other procedures to improve the classification?

• It is a supervened learning, the labels are there Commented Oct 27, 2016 at 6:00

## 2 Answers

For the imbalanced data you need to treat the classification task differently. For example monitoring your accuracy might tell you, you're doing well, although you never correctly classify the 5%. One option I used before was resampling, but I think there is good post in here and here.

• If I used Random Forest....are the approaches of the oversampling and the under sampling of the data needed? Commented Oct 27, 2016 at 8:26

When using Random Forests (or any other ensemble method), you can use one of the following (or both) approaches:

### Adapt the prior probabilities

Use this option if some classes are under- or overrepresented in your training set. Most training algorithms can handle this.

### Adapt the missclassification cost

If classes are adequately represented in the training data but you want to treat them asymmetrically, use the cost parameter $C$. Given you have a two class problem:

$$C = \left[ \begin{matrix} 0 & c_{0,1} \\ c_{1,0} & 0 \end{matrix}\right]$$

where $c_{i,j} > 1$ is the cost of missclassifying class $i$ as $j$. Note, that the penalty on the diagonal is 0, because it means that the classification was correct, i.e. $i = j$.

### Adapt the Training Algorithm

You can emply the RUSBoost Algorithm by Seiffert et al., which can handle imbalanced data pretty well.

### Further Reading

Matlab has a great section on this. You can find it here.