Fluctuating Validation Loss and Accuracy while training Convolutional Neural Network

Question

I am training a convolutional neural network with 3 layers to classify cancer cell images into one of the 2 classes. I am using ReLU activations to introduce non linearity and batchnorm / dropout per layer for regularization. I have 2340 images per class in my training set and 200 images per class in validation set (Both sets have perfectly balanced classes).

Even after enough regularization, it seems my model is still overfitting. What could be other possible reasons for fluctuating validation loss and accuracy? Should I try a lower learning rate? (current lr = 0.001). What can I do to solve this issue?

Answer 1

You don't have a huge amount of samples. I don't know how many nodes are in your fully-connected layers, how many parameters you have in total, or how large your images are, but it could be you just have way too many parameters for the number of samples you have. So you could try to change the structure of your network (e.g. lower the size of the convolutional layers or lower the size of the fully-connected layers) in order to lower the number of parameters that must be trained and increase dropout.