My CNN training gives me weird validation accuracy result. When it comes to 2.5,3.5,4.5 epochs, the validation accuracy is higher (meaning only need to go over half of the batches and I can reach better accuracy. But, If I go over all batches (one epoch), the validation accuracy drops). I repeat this experiment several times with random subset of data and the result looks similar.
Anything wrong here? When the accuracy is fluctuating? Also, when half cycle of epoch give better accuracy?
I use adadelta to train my network