2 classfiers, the better one has a higher CrossEntropy loss values in during training I have two classifiers (linear1 and linearGP). LinearGP has a better accuracy but it's CE loss has higher values in comparison with CE values of linear1.
linearGP is learned by another loss.
Data set is balanced. X axis represent samples during training process, at the end of the traning 30000 samples were passed through both models.  

What is the reason?
I think that one model returns very high probabilities for it's prediction whereas the other one doesn't although it is better in it's predictions
I created a simulated jupyter notebook example: https://github.com/cherepanovic/omwtuss/blob/master/CE_Acc_sim.ipynb
Would you agree or do you have also other arguments?
Thanks a lot!
 A: First, accuracy can be a poor choice for building or evaluating a model. When you say that "linearGP has a better accuracy" that doesn't necessarily mean it's the "better one."
Second, from your comments it's clear that what you are plotting is training error. An overfit model could well have a lower training error but a higher test error. So the model with the lower training error is not necessarily the "better one," either.
Third, it can be good to consider different loss functions for training as you evidently have done. The choice of loss function might differ depending on how you intend to use your model; the last half of this answer gives a brief overview. Make sure that your loss function provides a proper scoring rule. That said, it's not clear why the linearGP model differs from the linear1 model if linearGP was based on KL divergence and linear1 is based on cross-entropy, as these are the same. Perhaps the models involve different predictors, but that's not clear from the question.
