Oscillating validation accuracy for a convolutional neural network? My CNN training gives me weird validation accuracy result. When it comes to 2.5,3.5,4.5 epochs, the validation accuracy is higher (meaning only need to go over half of the batches and I can reach better accuracy. But, If I go over all batches (one epoch), the validation accuracy drops). I repeat this experiment several times with random subset of data and the result looks similar. 
Anything wrong here? When the accuracy is fluctuating? Also, when half cycle of epoch give better accuracy?
I use adadelta to train my network

 A: This is likely due to the ordering of your dataset. If there's many observations of the same class in a sequence the weights of the network will move too far in the direction of classifying this class.
A common cause is if you balance the classes in your dataset by resampling observations and appending them to the dataset. Shuffle your dataset - that should help you avoid the fluctuations in accuracy (and perhaps obtain a higher accuracy overall).
A: I had the same issue in the past and found out that the learning rate usually is the cause of oscillation. 
Try lowering your learning rate or using learning rate decay and keep training until the curve converges.  
A: Probably the learning rate is too high. The system is overfitting almost immediately, as overall the accuracy is falling after the first epoch.
If you want to find the sweet spot using early stopping you surely need a lower learning rate to extend your choice.
In addition, as suggested in other answers, I would use a learning rate scheduling.
Moreover, you may have a look at the size of your gradients.
Exploding gradients may cause this kind of oscillation.
Than using gradient clipping may be useful
