What is a good binary_crossentropy or categorical_crossentropy? I am training a binary classification model using LSTM and the training binary_crossentropy loss went from 0.84 to 0.83. I want to know what is a good binary_crossentropy loss value? There seems to be not many materials on the internet about this.
Besides, I am considering to change the problem into a 3-class classification problem. I also want to know what is a good categorical_crossentropy for multi-class classification problems?
 A: You’re not going to like this (neither do I), but, out of context, it’s impossible to say. In an easy pattern recognition problem
Like MNIST digit classification, you might demand an extremely low loss value. For hard problems where you know to expect less success (e.g., medical diagnosis or financial predictions), you might be able to have a useful model despite weaker performance.
I have a few guidelines.

*

*You should be able to beat a naïve model that always predicts based on the prior. In your binary problem has a $true:false$ class ratio of $3:1$, you should be able to do better than a model that always predicts $p(true)=0.25$. If you can’t outperform this, your model can’t do much. This is related to how $R^2$ compares your performance to the performance of a model that always predicts the same value.


*Compare your performance to that of rival models. If a rival model that is considered to have good performance gets a loss value of $0.5$, then maybe your loss value of $0.51$ is pretty good. Perhaps implementing your model is cheaper and makes up for the weaker performance; maybe that difference is not statistically significant. Likewise, if you achieve a loss value of $0.49$, your model might be state-of-the-art!
The takeaway: context is key.
