I have a difficulty understanding the intuition behind the logloss function since it seems to totally ignore negative examples where y = 0.
The images below visualize my question to some extend:
Your advice will be appreciated!
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ I don't know what you are trying to do, however read en.wikipedia.org/wiki/… for the cross entropy for logistic regression. Your loss is not specified correctly if this is what you intended to do. $\endgroup$– Cagdas OzgencCommented Nov 27, 2018 at 8:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The formula you used, seems to be
$$ H(X) = -P(X)\log P(X), $$
the definition of entropy. You seem to be asking about cross-entropy loss, also known as log-loss, which is defined as
$$ L(y, \hat y) = \underbrace{-y \log(\hat y)}_{\text{when } y=1} \;\underbrace{- (1-y) \log(1-\hat y)}_{\text{when } y=0} $$
where $y \in \{0, 1\}$ is the label and $\hat y$ is the predicted probability for the label. So the loss is zero for perfect classifications $y = \hat y = 1$, or $y = \hat y = 0$, and logarithmically increases otherwise.
