High AUC and Accuracy but weird output in confusion matrix

Question

I am working on image classification problem to determine gender given a face. The dataset is located here gender face dataset on kaggle (link to my notebook). The class distribution is as follows.

 Training 1600 images belonging to 2 classes. 
 Validation 340 images belonging to 2 classes. 
 Test 340 images belonging to 2 classes.

I am using RESNET along with a few other layers and I am achieving high accuracy.

base = ResNet50(weights='../input/keras-pretrained-models/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',
             include_top=False,
             input_shape=(150,225,3))

inp = Input(shape=(150,225,3))
base_out = base(inp)
out = Flatten()(base_out)
out = Dense(256, activation='relu')(out)
out = Dense(1, activation='sigmoid')(out)
model = Model(inp, out)

I added this callback in order to view the auroc curve.

def auroc(y_true, y_pred):
    auc = tf.metrics.auc(y_true, y_pred)[1]
    K.get_session().run(tf.local_variables_initializer())
    return auc

model.compile(loss='binary_crossentropy',
              optimizer=rmsprop,
              metrics=['acc',auroc])

My results are ACC: .98 AUC: .9406

The problem is the results of my confusion matrix do not seem to align with the values above. Below is how I am computing the matrix along with the output.

Y_pred = model.predict_generator(test_generator,steps = 340/batch_size)
from sklearn.metrics import confusion_matrix
y_pred = Y_pred > 0.5

con_mat = tf.confusion_matrix(
    test_generator.classes,
    y_pred,
    num_classes=2,
    dtype=tf.int32,
    name=None,
    weights=None
)
with tf.Session():
   print('Confusion Matrix: \n\n', tf.Tensor.eval(con_mat,feed_dict=None, session=None))

I am using keras back by tensorflow and ImageDataGenerators to load my data.

Answer 1

Seems like over fitting. Your model is able to memorize the answers on the training set, but doesn't generalize well to your test set.

I can't quite tell from your example code, but it looks to me like the high AUROC and Accuracy are on the training set, whereas the confusion matrix is from predicting on the test set