New answers tagged cross-entropy
0
This guy does an excellent job of working through the math and explanations from intuition and first principles. Take a peek.
tl;dr
Hinge stops penalizing errors after the result is "good enough," while cross entropy will penalize as long as the label and predicted distributions are not identical. The choice of cross-entropy entails that we aiming ...
2
The formula which you posted in your question refers to binary_crossentropy, not categorical_crossentropy. The former is used when you have only one class. The latter refers to a situation when you have multiple classes and its formula looks like below:
$$J(\textbf{w}) = -\sum_{i=1}^{N} y_i \text{log}(\hat{y}_i).$$
This loss works as skadaver mentioned on ...
1
Thank @djs for the great answer. Agreeing majority of it, but maybe not the last part. (Had to post another answer due to lack of reputation to comment directly.)
Another interesting property: suppose that there are three categories, with the first one being correct. Cross-entropy would value the predictions $(.8, .2, 0)$ and $(.8, .1, .1)$ equally, whereas ...
0
Does it ever make sense to use the form in equation (2) over equation (1) given that it has twice the number of parameters?
As far as I see, I have not found any arguments for doing this as stated in this question. With equation (2) you get:
an infinite number of solutions with almost all neural network architectures.
at worst (the binary specification and ...
Top 50 recent answers are included
