0
$\begingroup$

My CNN model has about 96~97% accuracy on both training and validation sets. But when submitting the test set it got only 24% accuracy. Here's my model:

def build_cnn_model():
    classifier = Sequential()
    classifier.add(Convolution2D(32, (3, 3), input_shape=(64, 64, 3), activation='relu'))
    classifier.add(MaxPooling2D())
    classifier.add(Convolution2D(32, (3, 3), activation='relu'))
    classifier.add(MaxPooling2D())
    classifier.add(Convolution2D(64, (3, 3), activation='relu'))
    classifier.add(MaxPooling2D())
    classifier.add(Flatten())
    classifier.add(Dense(64, activation='relu'))
    classifier.add(Dropout(0.5))
    classifier.add(Dense(4, activation='softmax'))

    classifier.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

    return classifier

The training set has about 40k images, valid set has about 10k images and test set is made of 5.5k images. Here's my implementation

train_datagen = ImageDataGenerator(rescale=1./255)
valid_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)

training_set = train_datagen.flow_from_directory(
    'datasets/training_set',
    target_size=(64, 64),
    batch_size=32,
    seed=42,
    class_mode='categorical')

valid_set = valid_datagen.flow_from_directory(
    'datasets/valid_set/',
    target_size=(64, 64),
    batch_size=32,
    seed=42,
    class_mode='categorical')

test_set = test_datagen.flow_from_directory(
    'original_data/',
    classes=['test'],
    target_size=(64, 64),
    seed=42,
    class_mode=None,
    batch_size=1)

test_set.reset()

classifier = build_cnn_model()

classifier.fit_generator(
    training_set,
    epochs=10,
    steps_per_epoch=1222,
    validation_data=valid_set,
    validation_steps=305)

Here we can see the behavior of the model acc_loss_cnn during training and I noticed that validation accuracy is always higher than training accuracy. So why this is happening? Why so low accuracy on test set and why validation accuracy is higher than training accuracy? What are the possible solutions?

$\endgroup$
  • $\begingroup$ Your test generator has class_mode=None, which differents from train/val. Is that intentional? $\endgroup$ – jonnor May 23 at 11:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.