What balancing method can I apply to an imbalanced data set?

Question

I'm trying to solve one classification problem from the UCI database repository. Unfortunately (or fortunately), I've noticed that my dataset is imbalanced. I've structured the data as 5 classes, according to the final mark reached by the pupil, like so:

• If student gets a mark from 0 to 7 => class 1 [FAIL(E)]
• If student gets a mark from 8 to 9 => class 2 [SUFFICIENT(D)]
• If student gets a mark from 10 to 11 => class 3 [GOOD(C)]
• If student gets a mark from 12 to 15 => class 4 [NOTABLE(B)]
• If student gets a mark from 16 to 19 => class 5 [OUTSTANDING(A)]

My problem is that, as I said, data are imbalanced, so I want to balance it.

I've thought about applying some kind of undersampling method, but my dataset has only 649 instances so I think removing some of them is not the best idea. Then I thought about doing some oversampling, in order to replicate some of the minority class examples and then get classes balanced, but I'm still unsure if that could work.

I would be very grateful if you could give me a hand with this. Its the first time I face a real problem with imbalanced data.

Answer 1

Since you're using R, you could make use of some elaborated methods like ROSE and SMOTE. But I'm not enrirely certain if re-balancing your dataset is the right solution in your case.

An alternative could be a cost-sensitive algorithm like C5.0 that doesn't need balanced data. You could also think about applying Markov chains to your problem.

Answer 2

I think that in your dataset the main challenge is not being imbalanced. The dataset is small and due to the few classes you don't have too many samples for any of them.

By using one-vs-all concepts ( A or not A, B or not B) you can get more samples for each concept. You can take advantage of the fact the the classes are ordered (A > B > C > D > E) and use a concept the aggregates some of them (e.g., B and above, D and bellow). Assuming there is no real difference in the reason to get D or E, not only you will get more samples, you also gain by diminishing the distinction between quite similar concepts.

As for changing the dataset in order to cope with the imbalance, go for it. However, you should validate on the original distribution. For details see: https://datascience.stackexchange.com/questions/810/should-i-go-for-a-balanced-dataset-or-a-representative-dataset/8628#8628

Instead of just over/under sampling, you can use a better technique in order to cope with the imbalance. For details see: https://www.quora.com/In-classification-how-do-you-handle-an-unbalanced-training-set/answer/Dan-Levin-2

Answer 3

Learning from Imbalanced Data

If you choose to oversample, be sure to do it after you create your train-test splits. If you are using cross validation, you should oversample within each fold.