There is the usual cross-entropy cost function: enter image description here

And then there is this cost function i stumbled upon in this paper: $$-\sum_{a}\sum_{k}ln[(O_{k}^{a})^{t^{a}_{k}}(1 - (O_{k}^{a} )^{1 - {t^{a}_{k}}}]$$

where $O$ are the activations and $t$ are the target values. What is the difference between these two?

  • $\begingroup$ They are the same. Just different notation. The h's correspond to the O's. i -> a, k -> k. $\endgroup$
    – jpmuc
    Commented Jun 7, 2020 at 13:39
  • $\begingroup$ @jpmuc is there an added benefit of using the shorter variation? $\endgroup$
    – mojbius
    Commented Jun 7, 2020 at 13:46

1 Answer 1


One is an average while the other is a sum. Otherwise, they're the same, just using different symbols.

Because they're the same (aside from the scalar $\frac{1}{m}$), there's no mathematical distinction between one and the other. Doing arithmetic on a computer would require special care, because floating point calculations can be fragile.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.