I need to build a text classification model.

I have a labeled training set and my goal is to classify the new unlabeled text .

My training set is composed on 6 categories, that are imbalanced.

The categories are distributed as follows: Category 1 -> 450 examples Category 2 -> 400 examples Category 3 -> 250 examples Category 4 -> 150 examples Category 5 -> 100 examples Category 6 -> 50 examples

How to deal with such imbalanced multi class text classification?


Generally, you should:

  • Sampling
  • Adjust your performance metrics (like F1 rather than just accuracy)
  • Choose a cost-sensitive algorithm, for example, adding weights to the minority classes
  • Algorithms such as decision tree, boosting etc. They are more adopted to imbalanced data set.
| cite | improve this answer | |
  • 1
    $\begingroup$ Could you provide insight into when which option should be chosen or why? With that addition this would be a great answer. $\endgroup$ – Nikolas Rieble Feb 24 '17 at 11:00

It really depends on your data but there are at least four things you could try:

  • Upsample the training set by copying the examples in each category
  • Downsample the training set by deleting some examples from the dominating categories
  • Use a boosting algorithm like scikit-learn's Adaboost
  • Use cost-sensitive classification algorithm
| cite | improve this answer | |

You can downsampling or upsampling but these are vary naive approaches. You could try to intelligently sample the data in order to "re-balance" the dataset. However, in any case: imbalanced data is very tricky and requires a lot of domain knowledge.

  • Always measure accuracy with the AUC: This denotes the probability that you can correctly classify a random positive instance and a random negative instance. There is no issue of imbalance here.

  • Coevolution with the genetic algorithm works well. You are evolving sub-models to collaborativley work together to solve the given task at hand.

  • Bayesian methods also work well here. Provided that you have a good prior distribution

Honestly - that's the best I can think of. Imbalanced classes are lame.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.