I have a convolution neural network with random weights initialized and Trained to perform binary classification. I have 2000 images as training data and 2000 validation data. The problem I am trying to solve is if the image is healthy or not.The loss function used is categorical cross entropy.

I have implemented the model in tensorflow.(tf.nn.sparse_softmax_cross_entropy_with_logits) The validation loss for that model with random weights is 0.71 and an accuracy was 58%. However, the same architecture as above initialized with a pre-trained weights gives a loss of 0.79 and accuracy 65% (Both the models converged without overfitting)

Does any one have any idea on why the loss in the second case is high yet has a better accuracy compared to the first one?

To put it another way, what is the contributing factor to the discrepancy between classification accuracy and categorical cross entropy, where a higher cross entropy corresponds to a better classification accuracy?

The learning curves for the both the cases are shown. learning curve for random weights learning curve for pretrained weights

  • $\begingroup$ I think that if you want people to give a reasonable explanation you need to supply more details. $\endgroup$ – Michael Chernick Apr 30 '17 at 16:28
  • $\begingroup$ I edited my question. Do you suggest adding more information? $\endgroup$ – Akshata Bhat Apr 30 '17 at 16:38
  • $\begingroup$ What variables are you using? What is the real problem you are trying to address? How much data do you have? How do you know that you are not overfitting in either case? $\endgroup$ – Michael Chernick Apr 30 '17 at 16:52
  • $\begingroup$ I have added the learning curves and a better description. I would be happy to hear your suggestions $\endgroup$ – Akshata Bhat Apr 30 '17 at 17:25
  • $\begingroup$ What do you mean by pre-trained weights in this case? Where are the non-random initialized matrices coming from? $\endgroup$ – user2879934 Apr 30 '17 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.