I was looking around for a good explanation as to why LSTMs are better able to handle vanishing and exploding gradients compared to vanilla RNNs. I know it is due to the cell memory $$c_t$$ acting as a constant error carousel (CEC): $$c_t=f_t\circ c_{t-1}+i_t\circ\widetilde{c}_t$$ So when I compute the gradient of the loss at time step $$t$$ w.r.t. a weight $$W$$ other than from the output gate, I get: $$\frac{\partial L_t}{\partial W}=\frac{\partial L_t}{\partial\hat{y}_t}\frac{\partial\hat{y}_t}{\partial h_t}\frac{\partial h_t}{\partial c_t}\left(\prod_{i=2}^t\frac{\partial c_i}{\partial c_{i-1}}\right)\frac{\partial c_{1}}{\partial W}.$$ If I made a mistake here please let me know. So my question is in regards to $$\frac{\partial c_t}{\partial c_{t-1}}$$, given the equation for $$c_t$$, it should be: $$\frac{\partial c_t}{\partial c_{t-1}}=f_t.$$ But I read a post that derived this expression: \begin{align*} \frac{\partial C_t}{\partial C_{t-1}} &= \frac{\partial C_t}{\partial f_{t}}\frac{\partial f_t}{\partial h_{t-1}}\frac{\partial h_{t-1}}{\partial C_{t-1}} + \frac{\partial C_t}{\partial i_{t}}\frac{\partial i_t}{\partial h_{t-1}}\frac{\partial h_{t-1}}{\partial C_{t-1}} \\ &+ \frac{\partial C_t}{\partial \widetilde{C}_{t}}\frac{\partial \widetilde{C}_t}{\partial h_{t-1}}\frac{\partial h_{t-1}}{\partial C_{t-1}} + \frac{\partial C_t}{\partial C_{t-1}} \end{align*} https://weberna.github.io/blog/2017/11/15/LSTM-Vanishing-Gradients.html