# CNN for multi-class classification with occasional multi-labels

I have about 10 classes, on which I train a CNN with a softmax output layer using one-hot encoding and categorical cross-entropy loss.

The problem is that two pairs of these of these classes (let's say A and B, and also E and F) also have very few "in between" examples. So I would expect the following possible scenarios:

• Input clearly belongs to a class from A to J
• Input belongs to A,B or input belongs to E,F

I see three options here.

• Introduce distinct output classes A&B, E&F. The issue is that I have relatively few examples here (possibly fixable), but I'm mainly worried about possible confusion - would the classifier be able to differentiate between A, B, A&B?
• Add training examples where the expected feature vector is [0.5, 0.5, 0, ...]. Theoretically this would make the ouptut close to 0.5 for the classes of interest, which I'm happy with.
• Change the output from softmax to sigmoid and treat it as a fully multilabel classification. I'm a bit worried that this would impact the overall accuracy, considering that I expect less than 5% of the inputs to be "in between".

Theoretically, I'd say all my options would work, but I can't tell their pros and cons.

• Many thanks for your answer. I'd like just a little clarification - at the very last, you say  if you randomly assign bats to the two classes. So, if I have an image with a bat, should I treat it as two training cases, labelled "bird" and "mammal", or have the target vector [0.5, 0.5, 0]? Jun 24 at 10:08