0
$\begingroup$

There is the usual cross-entropy cost function: enter image description here

And then there is this cost function i stumbled upon in this paper: $$-\sum_{a}\sum_{k}ln[(O_{k}^{a})^{t^{a}_{k}}(1 - (O_{k}^{a} )^{1 - {t^{a}_{k}}}]$$

where $O$ are the activations and $t$ are the target values. What is the difference between these two?

Improve this question
$\endgroup$
2
  • $\begingroup$ They are the same. Just different notation. The h's correspond to the O's. i -> a, k -> k. $\endgroup$
    – jpmuc
    Jun 7, 2020 at 13:39
  • $\begingroup$ @jpmuc is there an added benefit of using the shorter variation? $\endgroup$
    – mojbius
    Jun 7, 2020 at 13:46

1 Answer 1

Reset to default
0
$\begingroup$

One is an average while the other is a sum. Otherwise, they're the same, just using different symbols.

Because they're the same (aside from the scalar $\frac{1}{m}$), there's no mathematical distinction between one and the other. Doing arithmetic on a computer would require special care, because floating point calculations can be fragile.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.