CNN for multi-class classification with occasional multi-labels

Question

I have about 10 classes, on which I train a CNN with a softmax output layer using one-hot encoding and categorical cross-entropy loss.

The problem is that two pairs of these of these classes (let's say A and B, and also E and F) also have very few "in between" examples. So I would expect the following possible scenarios:

• Input clearly belongs to a class from A to J
• Input belongs to A,B or input belongs to E,F

I see three options here.

• Introduce distinct output classes A&B, E&F. The issue is that I have relatively few examples here (possibly fixable), but I'm mainly worried about possible confusion - would the classifier be able to differentiate between A, B, A&B?
• Add training examples where the expected feature vector is [0.5, 0.5, 0, ...]. Theoretically this would make the ouptut close to 0.5 for the classes of interest, which I'm happy with.
• Change the output from softmax to sigmoid and treat it as a fully multilabel classification. I'm a bit worried that this would impact the overall accuracy, considering that I expect less than 5% of the inputs to be "in between".

Theoretically, I'd say all my options would work, but I can't tell their pros and cons.

Answer 1

The disadvantage of the first option (additional classes) is that you do not use the information you learn from "clear-cut" instances to classify "in-between" instances. Especially if there are few of the latter, this can seriously harm the performance of your model.

The third alternative (multilabel classification) may suffer from the same problem as the first alternative (too few multilabel instances). By the way, one of the main methods of multilabel classification is the first option you propose.

The second alternative seems perfectly coherent with the fact that you have no prior about the right label for multilabel instances. Moreover, it uses data efficiently: the knowledge acquired from "clear-cut instances" is transferred to "in-between instances".

A fourth possibility, which is equivalent to the second one, but probably easier to implement (no tweaks to the standard routines used for image classification) is to randomly assign "in-between" instances to one of the two "clear-cut" classes. Possibly, you could randomly re-assign "in-between" instances at each epoch of the training process.

For example, suppose that your model has three classes: mammals, bird, fish. Moreover, you have few images with bats, and you are uncertain whether to classify bats as mammals or birds (they are mammals, but let's suppose). If you create a distinct class (mammal-bird), you will hardly be able to learn something useful about that class. However, if you randomly assign bats to the two classes, the model will probably be able to recognize wings (pattern learned from birds) and mouse-like features (pattern learned from mammals).