# Understanding cross entropy in neural networks [duplicate]

This article provides a nice explanation and derivation of cross entropy, and defines it as follows:

$H(y, \hat{y}) = -\sum_i[ y_i \log \hat{y}_i]$ ....... (1)

where $y$ is the correct output and $\hat{y}$ is the network's predicted output.

Many neural network articles use this definition of cross entropy:

$Loss = -\sum_i[ y_i \log \hat{y}_i + (1-y_i) \log(1- \hat{y}_i))]$ ....... (2)

These equations are different, where does equation 2 come from? Is equation 2 a better loss function for neural networks?

## marked as duplicate by Sycorax, mdewey, jld, kjetil b halvorsen, JohnAug 8 '17 at 16:58

If $i \in \{1,0\}$ these equations are identical. The second is commonly used for the case of a binary response whereas the first is a more general form used for $k$ classes.