# Understanding difference between gradient of sigmoid vs softmax in the context of back propagation

Consider the step by step backpropagation shown in this article. The neural network given there is: The outer layer employs sigmoid. It also employs mean squared error loss function. During back-propagation, it calculates following gradient:

$$\frac{\partial E_{total}}{\partial w_5} = \color{red}{\frac{\partial E_{total}}{\partial out_{o1}}} \times \color{green}{\frac{\partial out_{o1}}{\partial in_{o1}}} \times \color{blue}{\frac{\partial in_{o1}}{\partial w_5}}$$ $$=\color{red}{-(y_{o1}-out_{o1})} \times \color{green}{out_{o1}(1-out_{01})} \times \color{blue}{out_{h1}}$$

Now, for softmax function $$y$$, $$\frac{\partial y_i}{\partial s_k} =\begin{cases} \color{green}{y_i(1-y_i)}, & \text{if i=k} \\ -y_iy_k, & \text{if i\neq k} \end{cases}$$

I am confused with what will happen when outer layer employes softmax. Will the partial derivative remain same as before:

$$\frac{\partial E_{total}}{\partial w_5} = \color{red}{\frac{\partial E_{total}}{\partial out_{o1}}} \times \color{green}{\frac{\partial out_{o1}}{\partial in_{o1}}} \times \color{blue}{\frac{\partial in_{o1}}{\partial w_5}}$$ $$=\color{red}{-(y_{o1}-out_{o1})} \times \color{green}{out_{o1}(1-out_{01})} \times \color{blue}{out_{h1}}$$

because, $$i=k=o1$$ in $$\color{green}{\frac{\partial out_{o1}}{\partial in_{o1}}}$$ or the term $$-y_i y_k$$ also appear somewhere? I am confused, because this video shows how $$-y_iy_k$$ should also appear, at the end, say at time 16:44: PS: Sorry if cropped image at the end looks bad, I just wanted to point to what I was talking about in the video.