Choosing the number of labels in a multiclass classification problem

Question

I've recently come accross a multilabel classification problem. Here, multiple labels can be simultaneously assigned to a single instance.

I am interesting how one determines the number of labels to be predicted on an instance level. Normally, if only one label per instance is given, one does:

$$ P_x = argmax (P_{i \dots j}) $$, i.e., the label with the highest probability is selected. What if multiple labels are possible (e.g., recommendation systems, where a person can like multiple movies?).

I understand, that a neural network, for example, already yields a vector and one could do $$P_x > 0.5$$, yet this does not seem as the best option, or is it?

Thank you.

Answer 1

Here are a couple of options.

1.) You can treat each label prediction separately. I.e. if the label set is $L$, then learn a mapping $f : X\rightarrow \mathbb{R}^{|L|} $. For the final outputs, use a sigmoid non-linearity, so that each element of $f(x)$ is in $[0,1]$. Then you can use the sum of binary cross entropies as the loss function. To count the number of labels, yes, you need to define a threshold. This is exactly the approach you are suggesting in your own question, and I think it is fine.

2.) If you don't have too many labels and/or some labels are mutually exclusive, you can redefine outputs to make this into a regular multi-class problem (rather than multi-label). For instance, if the options are $A$, $B$, $A$ and $B$, and neither, then you can view this as a 4-class problem. Then the number of classes is determined by the output of the classifier (e.g. the classes have 1, 1, 2, and 0 labels respectively, in the example above).

3.) You could actually learn a model to predict the number of labels directly. This can be done via e.g. a classifier, by taking the number of classes to be the maximum possible number of labels (plus one if no labels is an option).

Related questions: [1], [2], [3], [4], [5]