# How low does the cross entropy loss need to be for me to be confident in my model?

I am training a neural network for a multi-class classification task. The loss function I am using is the CrossEntropyLoss implemented in pytorch, which is, according to the documents, a combination of logsoftmax and negative log likelihood loss (forgive me for not knowing much about them, all I know is that cross entropy is frequently used for classification). I started out with a loss of 2.4 and then it descends to and fluctuates in the range 0.65~0.7. Can I infer from this value how accurate my model is now? I remember reading in an article that a cross entropy loss of 0.69 is equal to random guess in the binary classification case, is it different somehow in the multi-class classification? Or does it mean my model is basically guessing randomly?

More details, if they matter:

• I am running on CIFAR10, so there are 10 classes.

• The loss is calculated per sample, so the batch size is not reflected here.

is it different somehow in the multi-class classification? Or does it mean my model is basically guessing randomly?

You have to understand how you get the score of a random baseline.

Suppose you have a binary classification problem where classes are balanced, that is, the pmf of the labels in your dataset is $$q_i=1/2$$. If you do random guesses, you don't even look at the inputs (images in your case) and assign probability $$p_i=1/2$$ to the two classes all the time. The cross-entropy you obtain is $$H[q,p] = - \sum_i q_i \log p_i = - 2 * 1/2 * \log(1/2) = \log(2) = 0.69$$ (using the natural log).

From there, you can generalise this reasoning in the 10-classes case by yourself. In the unbalanced case (CIFAR10 is balanced so you won't need that, but I'm still explaining), of course you simply change the $$q_i$$.

Understanding the above, you 1) understand why the model starts out with a loss around $$2.4$$ and 2) know whether you are doing better than guessing at random. Leave a comment if that is not the case.

Can I infer from this value how accurate my model is now?

Accuracy has a precise meaning in evaluation of classification model and I don't know if that's what you mean. You can not infer accuracy from the cross-entropy. Accuracy is defined as the proportion of labels that are correctly classified by your classifier on the test set. How do you "classify" using a probabilistic model? You choose the class that has the highest probability, this is called the Bayes classifier.