I want to build a convolutional neural network and train it to recognise whether the digit is 0 or 1.

Example of my training data is a 800 * 600 gray scale image containing a digit one:enter image description here I have 22 such images, 11 containing zeroes, 11 containing ones.

I now build a convolutional nerual network:

def addConvolutionalLayer(layer, newDepth = None):
    if newDepth is None:
        newDepth = layer.get_shape().as_list()[3] * 2
    newLayer = Conv2D(newDepth, kernel_size = (3, 3),
                activation = 'relu', padding='same') (layer)
    newLayer = Dropout(0.2) (newLayer)
    return Conv2D(newDepth, kernel_size = (3, 3),
        activation = 'relu', padding='same') (newLayer)

def buildModel():
    s = Input((600, 800, 1))
    d1 = addConvolutionalLayer(s, 32)
    d2 = addConvolutionalLayer(MaxPooling2D(4)(d1))
    d3 = addConvolutionalLayer(MaxPooling2D(4)(d2))
    d4 = addConvolutionalLayer(MaxPooling2D(4)(d3))
    f = Flatten()(d4)
    dense = Dense(64, activation='relu') (f)
    o = Dense(1, activation='sigmoid') (dense)
    model = Model(inputs = [s], outputs = [o])
    model.compile(optimizer='adam', loss='binary_crossentropy',
    return model

model = buildModel()

In short, a network consists of multiple convolutional layers (with MaxPooling included), ending with a dense layer.

I then split the data into training and test sets:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

X_train, X_test are arrays of float values, each value corresponding to the pixel in the image (1.0 for white, 0.0 for black).

y_train, y_test are lists containing 0 or 1 depending on the digit in the corresponding image.

Finally, I run the model to train:

model.fit(X_train, y_train, epochs = 50, validation_split=0.2)

The problem is that the model gets over-fitted: it reaches 100% accuracy on the training data, but does not perform well on the validation set:

Epoch 95/100
12/12 [==============================] - 0s 32ms/step - loss: 1.0640e-07 -
    acc: 1.0000 - val_loss: 10.7454 - val_acc: 0.3333

How to fix this issue?


First thing you could try is that remove Dropout in ConvBase and add before dense layers (need to experiment a bit on dropout percentage value), then experiment with different pre-processing techniques to augment the data. Most importantly check no.of parameters by getting summary of network and try to reduce the no.of parameters in case if no.of parameters are high.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.