3
$\begingroup$

For multiclass classification, does the negative log likelihood loss function only take the loss for the classification group? i.e

$$ C(\theta) \equiv \sum{}{}y_ilog(\hat{y}_i) $$

Doesn't $y_i$ just go to zero for all groups except the actual group a given training sample belongs to?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Yes, it does. It could have been written as $$C(\theta)=\log \hat y_{c(i)}$$ where $c(i)$ equals to the actual class of the i-th sample, but then differentiating the loss with respect to parameters wouldn't be straightforward as in here.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Why does the binary version take into account both the loss for the correct and incorrect classification group? $\endgroup$
    – James Ellis
    Commented Mar 22, 2020 at 18:38
  • 1
    $\begingroup$ Typically in multiclass classification problems, there are $c$ outputs, and we choose maximum of them as answer. But, in binary classification problems, you have only one output instead of two outputs (that's the way the classifier is typically designed) as if $c=2$. That's why the loss term includes as if there is a second output $\endgroup$
    – gunes
    Commented Mar 22, 2020 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.