# Convolutional neural network fails even when given answer

I was having problems with a CNN giving the prediction as true for everything regardless of input. Taking advice from this forum, I simplified the input to give it the output as the input and it's still unable to make the prediction correctly! Shape is 99,22, 2. The output boolean is in the input in the 3rd dimension of the input.

Here's an example of 1 sample of the input: https://pastebin.com/jCVU3brn to predict the output as 0.

def CNN(train_X, train_y, test_X, test_y):

model = Sequential([
Conv2D(30, kernel_size=3, activation="relu", input_shape=(99, 25, 2)),
Conv2D(64, kernel_size=3, activation="relu"),
Flatten(),
Dense(1, activation='softmax')
])

# Compile the model.
model.compile(
'adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)

# Train the model.
model.fit(
train_X,
train_y,
epochs=1
)
preds = np.round(model.predict(test_X), 0)

return preds


Model summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_11 (Conv2D)           (None, 97, 23, 30)        570
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 95, 21, 64)        17344
_________________________________________________________________
flatten_4 (Flatten)          (None, 127680)            0
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 127681
=================================================================
Total params: 145,595
Trainable params: 145,595
Non-trainable params: 0


## 1 Answer

Your last layer is a fully connected layer with 1 output unit and softmax activation. The softmax activation takes vector $$x$$ as input and computes a vector $$y$$ of outputs, where $$y_i = \frac{e^{x_i}}{\sum_{j=1}^D e^{x_j}}$$ and $$D$$ is the dimension of the input. In your case, since the dimension of the output is 1, this reduces to:

$$y_1 = \frac{e^{x_1}}{e^{x_1}} = 1$$

So you've trained a classification network, but the last layer implies there's only 1 valid category, which means the network must assign probability 1 to that category. It sounds like it's the case that you have more than one category, in which case you should use more than 1 output unit.

• There is only one output category. It is either true or false. The output is always 0. Aug 16, 2020 at 6:29
• if you have "true" and "false", then that is two categories. you have two categories, but you're only allowing your network to output a single category, which is why your results are nonsense Aug 16, 2020 at 16:39
• OK thanks but there is really no way to output a single binary result for just one category in a CNN? 0 = not the category, 1 = is the category? Is the problem that it is softmax? Aug 18, 2020 at 7:22
• If I understood the answer correctly, all you need to do is increase the dimensions of the last layer to 2: Dense(2, activation='softmax') Although you could use a sigmoid function instead - see stats.stackexchange.com/questions/218542/… Aug 20, 2020 at 12:20