# Help me interpret my VGG16 fine-tuning results

I have a binary classification problem where I'm trying to classify whether a given cell is cancerous or not. For this I decided to play around with VGG16 pre-trained model and simply remove the last dense layer which is used to classify the 1000 classes required for the imagenet and instead make it 2 class dense with softmax as activation.

How the model looks:

The problem:

Looking at the tensorboard where I compare 2 models that I've trained I can't seem to make sense of the results. It seems that after 30 epochs of training my validation loss begins increasing, but validation accuracy stays static, what gives? surely as my predictions begin deviating, the accuracy should change because the deviation should cause enough change on the weights to cause softmax to begin missclassifying images. Looking at my tensorboard it seems not to be the case.

The only way this makes sense is that if my validation set contains only of images of the same class and the model is constantly just guessing the same thing and just by chance it's guessing the right class constantly, but I checked my folder structure and the images are categorised as following:

positive-negative
|
+--test:
|   +--negative:77
|   +--positive:77
+--train:
|   +--negative:154
|   +--positive:154
+--valid:
|   +--negative:26
|   +--positive:26


This is how my batches are setup:

train_batches = ImageDataGenerator(rescale=1./255).flow_from_directory(fp_train, target_size=(width,height), classes=["negative","positive"], batch_size=2)
test_batches = ImageDataGenerator(rescale=1./255).flow_from_directory(fp_test, target_size=(width,height), classes=["negative","positive"], batch_size=test_size)
valid_batches = ImageDataGenerator(rescale=1./255).flow_from_directory(fp_valid, target_size=(width,height), classes=["negative","positive"], batch_size=2)


And my fitting configuration:

VGG16_Sequential.fit_generator(train_batches,
steps_per_epoch=154,
validation_data=valid_batches,
validation_steps=26,
epochs=epochs,
verbose=1,
callbacks=[tf_board, cp_cb])


What could possibly explain these results? I checked the validation set images and they are out-of-sample from the training and I clearly have two distinct cateogories.