How to calculate the derivative of crossentropy error function?

Question

I'm reading this tutorial (presented below) on computing derivative of crossentropy. The author used the loss function of logistic regression I think. https://www.dropbox.com/s/rxrtz3auu845fuy/Softmax.pdf?dl=0

Most of the equations make sense to me except one thing. In the second page, there is: $$\frac{\partial E_x}{\partial o^x_j}=\frac{t_j^x}{o_j^x}+\frac{1-t_j^x}{1-o^x_j}$$ However in the third page, the "Crossentropy derivative" becomes

$$\frac{\partial E_x}{\partial o^x_j}=-\frac{t_j^x}{o_j^x}+\frac{1-t_j^x}{1-o^x_j}$$

There is a minus sign in $E_x$. Then the derivative should be $\frac{\partial E_x}{\partial o^x_j}=-\frac{t_j^x}{o_j^x}-\frac{1-t_j^x}{1-o^x_j}$. But it is not. What have I missed?

---

The tutorial:

Answer 1

There is indeed a mistake in slide with title "Crossentropy Error Function":\begin{align} \frac{\partial E_x}{\partial o_j^x} &=\frac{\partial }{\partial o_j^x} \left( - \sum_{k}[t_k^x \log(o_k^x) + (1-t_k^x) \log(1-o_k^x)]\right) \\ &=-\frac{\partial }{\partial o_j^x} \left( \sum_{k}[t_k^x \log(o_k^x) + (1-t_k^x) \log(1-o_k^x)]\right) \\ &=-\frac{\partial }{\partial o_j^x} \left( [t_j^x \log(o_j^x) + (1-t_j^x) \log(1-o_j^x)]\right) \\ &=- \left( \frac{t_j^x}{o_j^x} - \frac{1-t_j^x}{1-o_j^x}\right), \text{Chain rule} \\ &=- \frac{t_j^x}{o_j^x} + \frac{1-t_j^x}{1-o_j^x} \\ \end{align}

Answer 2

An easy way to remember this is to internalize the gradient of the cross-entropy with respect to network parameters, which is famously $t_i - o_i$.

The last slide does this correctly. So, it looks like the second slide has a mistake. If you follow the derivations you'll notice the mistake where for no reason a minus sign appears in the middle of the right hand side (before the last equation).