What is the possible cause of low validation loss and higher training loss? Also the accuracy is fluctuating. enter image description here

  • $\begingroup$ Usually it's the other way around. Is your validation set randomly drawn? $\endgroup$ – Chris Dec 12 '16 at 19:32
  • 2
    $\begingroup$ It's a guessing game until you tell us more... $\endgroup$ – Tim Dec 12 '16 at 19:32
  • 1
    $\begingroup$ please write more explicit titles $\endgroup$ – Franck Dernoncourt Dec 12 '16 at 19:44
  • $\begingroup$ The problem I'm trying to solve involves time series data. The entire data is sequential. I read that for time series data it's good not to randomize data to take advantage of temporal relationship between data. $\endgroup$ – Tata Dec 12 '16 at 20:11
  • $\begingroup$ @Tim, I'm trying to solve time series problem using CNN with three convolution layers. I tried L2 regularization, dropout and batch normalization but, I end up with low validation loss and high training loss. Validation accuracy is always fluctuating. $\endgroup$ – Tata Dec 12 '16 at 20:19

I assume that it may be an overfitting problem: high variance and low bias in your model. If you're working with neural network, maybe a good idea is to use a some kind of regularization (L1 or L2 depending on the model you're implementing) or reduce the number of layers if you have more than one hidden layer and units in it.

Also a good idea may be to try another output function (for instance, if you're using sigmoid, try to use tanh). There's no good common way to develop a network, it's all process of fail and trial, so I think that after some time of trying different models you'll improve the performance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.