4
$\begingroup$

My CNN training gives me weird validation accuracy result. When it comes to 2.5,3.5,4.5 epochs, the validation accuracy is higher (meaning only need to go over half of the batches and I can reach better accuracy. But, If I go over all batches (one epoch), the validation accuracy drops). I repeat this experiment several times with random subset of data and the result looks similar.

Anything wrong here? When the accuracy is fluctuating? Also, when half cycle of epoch give better accuracy?

I use adadelta to train my network

enter image description here

$\endgroup$
  • 1
    $\begingroup$ Were you able to figure out what was causing the issue? $\endgroup$ – Raghuram Jun 5 '18 at 8:45
  • 1
    $\begingroup$ @RockTheStar How big is your dataset? What kinds of data such as image, audio, video? Also what is output of your neural network? Did you try different batch size? $\endgroup$ – Cloud Cho Jul 23 at 16:04
5
$\begingroup$

This is likely due to the ordering of your dataset. If there's many observations of the same class in a sequence the weights of the network will move too far in the direction of classifying this class.

A common cause is if you balance the classes in your dataset by resampling observations and appending them to the dataset. Shuffle your dataset - that should help you avoid the fluctuations in accuracy (and perhaps obtain a higher accuracy overall).

$\endgroup$
  • $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? You can also turn it into a comment. $\endgroup$ – gung - Reinstate Monica Jan 8 '16 at 13:24
  • $\begingroup$ I believe it did provide an answer, but I've now expanded a bit on it :) $\endgroup$ – pir Jan 8 '16 at 13:42
  • $\begingroup$ I have tried to shuffle the data, but I think I occasionally got similar graph. $\endgroup$ – RockTheStar Jan 8 '16 at 18:50
1
$\begingroup$

I had the same issue in the past and found out that the learning rate usually is the cause of oscillation. Try lowering your learning rate or using learning rate decay and keep training until the curve converges.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.