I have a simple question which I cannot find a straight answer to. I am training a neural network to classify some medical images. I initially focused on validation accuracy after each epoch (to determine how the network was generalising) and then after that, test accuracy on an unseen dataset.

But I see that validation loss is also important - and sometimes my validation loss drops even thought for that epoch, the validation accuracy also goes down slightly. Ultimately test accuracy is going to be the gold standard I guess, but thats no use for guiding training and adjusting parameters. Obviously if the the validation loss starts to go up and validation accuracy starts to drop it indicates overfitting. But what about when both validation loss and validation accuracy drop after an epoch?

  • $\begingroup$ validation loss and validation accuracy are correlated, well, but not perfectly correlated. Perfect correlation is a myth! There could be one off (outlying) sample in your validation set, that shows this. $\endgroup$
    – Rajesh D
    Dec 21 '17 at 14:35
  • $\begingroup$ Ok, but the question is asking how one might interpret these in terms of which might best reflect generalizability of the algorithm. $\endgroup$
    – GhostRider
    Dec 21 '17 at 14:37
  • $\begingroup$ "loss is in your hand" (that's what you are optimizing!). "accuracy is God's business". $\endgroup$
    – Rajesh D
    Dec 21 '17 at 14:41

Expanding on my comment, The main goal of your training algorithm is to decrease the net loss. So as long as your validation loss is decreasing, your algorithm is doing good, and is generalizing well. But the validation accuracy is not improving or getting worse, consider relooking at the loss function you are optimizing, it indicates that the loss function is not suitable for the task, rather than training algorithm/model being defective. Go for some other loss function, and then select a model/algorithm that is suitable for that loss function.


This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .