# The Best Approach for the Classification of the imbalanced classes

I have data that I am going to classify to 3 classes, but one class has a few samples less than 5% of the total samples. What is the best approach to classify these imbalanced classes? I mean is there a machine learning technique that fits this case or is there any other procedures to improve the classification?

• It is a supervened learning, the labels are there – mhdella Oct 27 '16 at 6:00

For the imbalanced data you need to treat the classification task differently. For example monitoring your accuracy might tell you, you're doing well, although you never correctly classify the 5%. One option I used before was resampling, but I think there is good post in here and here.

• If I used Random Forest....are the approaches of the oversampling and the under sampling of the data needed? – mhdella Oct 27 '16 at 8:26

When using Random Forests (or any other ensemble method), you can use one of the following (or both) approaches:

Use this option if some classes are under- or overrepresented in your training set. Most training algorithms can handle this.

If classes are adequately represented in the training data but you want to treat them asymmetrically, use the cost parameter $C$. Given you have a two class problem:

$$C = \left[ \begin{matrix} 0 & c_{0,1} \\ c_{1,0} & 0 \end{matrix}\right]$$

where $c_{i,j} > 1$ is the cost of missclassifying class $i$ as $j$. Note, that the penalty on the diagonal is 0, because it means that the classification was correct, i.e. $i = j$.