How to manually balance unbalanced multi-class/multi-label data?

Question

I have a multi-class and multi-label classification problem, i.e.: each sample can have more than one label associated to it and there is a total number of M possible labels.

e.g.:

• y[0] = [0]
• y[1] = [0, 1]
• y[2] = [1, 4, 3, 0]
• y[3] = [0, 1]
• ...
• y[100] = [1, 0, 3]

Counting the number of occurrences of each label, I can see that some labels are way more frequent than others. In the example above, for instance, 0 appears more often than 1, 3 and 4.

I can't figure out a smart (over-)sampling strategy to have a dataset where each label appears approximately the same number of times.

Any papers/idea on that?

Answer 1

Since each sample can have more than one label associated with it one possible solution would be to train $M$ logistic regression models, where the response variable for label $i$, sample $n$, is $Y_{i,n}$ where $Y_{i,n}=1$ if sample $n$ belongs to class $i$ and $Y_{i,n}=0$ if it does not.

For a new sample $Z$ your model output can be interpreted as the probability $Z$ belongs to each individual class.