# CNN - I know in advance which class my input cannot be

I have a CNN built with Keras, based on a multilayer perceptron. The last layer is softmax.

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 248, 248, 32)      896
_________________________________________________________________
activation_1 (Activation)    (None, 248, 248, 32)      0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 124, 124, 32)      0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 122, 122, 32)      9248
_________________________________________________________________
activation_2 (Activation)    (None, 122, 122, 32)      0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 61, 61, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 59, 59, 64)        18496
_________________________________________________________________
activation_3 (Activation)    (None, 59, 59, 64)        0
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 29, 29, 64)        0
_________________________________________________________________
flatten_1 (Flatten)          (None, 53824)             0
_________________________________________________________________
dense_1 (Dense)              (None, 64)                3444800
_________________________________________________________________
activation_4 (Activation)    (None, 64)                0
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0
_________________________________________________________________
dense_2 (Dense)              (None, 9)                 585
_________________________________________________________________
activation_5 (Activation)    (None, 9)                 0
=================================================================
Total params: 3,474,025
Trainable params: 3,474,025
Non-trainable params: 0
_________________________________________________________________


Now, during classification, there are some cases in which I know in advance my input is not class A, or B, or C (because of some specific business logic).

Example : I have an image X that I want to classify. My CNN can classify between 10 different classes. I know, without doing anything, that my input is not class A or class B.

Is there any way I could theoretically take advantage of this knowledge to help the classification process ?

Since the last layer is in softmax, I was thinking of setting the values of the output neurons I know are not the right answer to 0 and redistribute this value among potentially right neurons.

I hope my question was clear enough.

• You could train a separate model that has only learned to distinguish between those two classes. Since it has a more specific goal, given enough examples it might outperform the three-classes model. – Frans Rodenburg Apr 26 at 11:46
• The thing is, class A or B were examples, but this could be different. At each classification, I might know in advance one or more classes that my input cannot be an instance of. But that's variable, and it could be different classes – Arthur Attout Apr 26 at 12:26

Typically you have $$P(y|x) = \text{softmax} (f(x;\theta))$$, where $$f$$ is your network. But given a mask $$m$$ of invalid classes, you can simply say $$P(y|x,m) = \text{softmax}(f(x;\theta) - km)$$, where $$k$$ is set to some very large constant. And this works fine both at train time and at test time.