0
$\begingroup$

I have data that I am going to classify to 3 classes, but one class has a few samples less than 5% of the total samples. What is the best approach to classify these imbalanced classes? I mean is there a machine learning technique that fits this case or is there any other procedures to improve the classification?

$\endgroup$
  • $\begingroup$ It is a supervened learning, the labels are there $\endgroup$ – mhdella Oct 27 '16 at 6:00
0
$\begingroup$

For the imbalanced data you need to treat the classification task differently. For example monitoring your accuracy might tell you, you're doing well, although you never correctly classify the 5%. One option I used before was resampling, but I think there is good post in here and here.

$\endgroup$
  • 1
    $\begingroup$ If I used Random Forest....are the approaches of the oversampling and the under sampling of the data needed? $\endgroup$ – mhdella Oct 27 '16 at 8:26
0
$\begingroup$

When using Random Forests (or any other ensemble method), you can use one of the following (or both) approaches:

Adapt the prior probabilities

Use this option if some classes are under- or overrepresented in your training set. Most training algorithms can handle this.

Adapt the missclassification cost

If classes are adequately represented in the training data but you want to treat them asymmetrically, use the cost parameter $C$. Given you have a two class problem:

$$C = \left[ \begin{matrix} 0 & c_{0,1} \\ c_{1,0} & 0 \end{matrix}\right] $$

where $c_{i,j} > 1$ is the cost of missclassifying class $i$ as $j$. Note, that the penalty on the diagonal is 0, because it means that the classification was correct, i.e. $i = j$.

Adapt the Training Algorithm

You can emply the RUSBoost Algorithm by Seiffert et al., which can handle imbalanced data pretty well.

Further Reading

Matlab has a great section on this. You can find it here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.