Why is the training accuracy and validation accuracy both fluctuating?

Question

I am currently fine tuning VGG16 network to do a binary classification task. I have to admit that the training and testing samples are relatively small (around ~60 for training and ~15 for testing). I have tried data augmentation techniques from keras modules (imagedatagenerator).

When I feed these data into the VGG16 network (~5 epochs), the network's training accuracy and validation accuracy both fluctuates as the figure below. Attached with figures showing the accuracies and losses.

May I know what does this phenomenon indicate? Many thanks!

Answer 1

You train your neural network with gradient descent, hence you will find only a LOCAL minimum for your choice of loss. Every time your run your neural network you typically initialize your net differently. Hence it is likely that you will end up in a different local minimum, which may be better or worse than the previous one...

This phenomenon just indicates that your optimization problem is not convex, hence has not a single minimum. You can try using a classifier that has a convex objective as e.g. Support Vector Machines.