Data augmentation or weighted loss function for imbalanced classes?

Question

I have a CNN image classification problem with imbalanced classes. I could balance the dataset using data augmentation (Replication, mirror, etc.) on the minority classes. Also, imbalance could be handled using a weighted loss function that gives more importance to the minority classes. Is any of these methods preferred over the other and if so, why?

Answer 1

I've tried both minority (random) oversampling and class weighting to solve the problem of imbalance in the past and none proved to be better than the other (which is logical). The reason why I'd suggest class weights is that it has a much less training time than the over-sampling, as the number of examples is much fewer.

I've never tried under-sampling, as it seems counter-intuitive due to the fact that CNNs benefit greatly from a larger dataset.

To be honest, I haven't thought of augmenting just the minority classes to balance them and in my opinion, given the right choice of augmentors, it could outperform both random over-sampling and class weighting.

However, by doing this you are missing on an important thing that could boost the performance of your CNN, augmenting the majority class. By augmenting just the minority classes, you would be missing out on a large number of training images that would have been generated by augmenting the majority class ones.

If I were to guess I'd say that class weighting would provide the best results, of the scenarios discussed, given you performed data augmentation on the whole dataset.

Answer 2

You could try with a combination or Oversampling + Undersampling (i.e. replication and removal or certain "less useful" data points; applying SMOTE(and or any of its variants) + bagging...). You could also apply an ImageGenerator to increase your sample size while balancing the classes.