4
$\begingroup$

I looked at the answer to this question https://stackoverflow.com/questions/41990250/what-is-cross-entropy to try to understand cross-entropy and it seems to me that when the true label for a class is 0, the loss wouldn't increase no matter what the prediction was because the log of the prediction would be multiplied by 0. This doesn't seem like a very accurate way to calculate loss to me. Am I missing something?

$\endgroup$

1 Answer 1

4
$\begingroup$

$$ \mathcal{L}(\theta)= -\frac{1}{n}\sum_{i=1}^n \left[y_i \log(p_i) + (1-y_i) \log(1-p_i)\right] $$ When $y_i=0$, the second term is nonzero for $p_i \in (0,1)$. The factors involving $y_i$ are like a "switch." Only one of the factors involving $y_i$ is nonzero, so that is the only relevant loss term for sample $i$.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.