# back propagation through time derivation issue

I read several posts about BPTT for RNN, but I am actually a bit confused about one step in the derivation. Given

$$h_t=f(b+Wh_{t-1}+Ux_t)$$

when we compute $$\frac{\partial h_t}{\partial W}$$, does anyone know why is it simply

$$\frac{\partial h_t}{\partial W}=\frac{\partial h_{t}}{\partial h_{t-1}}\frac{\partial h_{t-1}}{\partial W}$$

not

$$\frac{\partial h_t}{\partial W}=\frac{\partial h_{t}}{\partial h_{t-1}}\frac{\partial h_{t-1}}{\partial W}+\frac{\partial h_t}{\partial f}h_{t-1}$$

?

What I mean is, since both $$W$$ and $$h_{t-1}$$ depends on $$W$$, why is the second term in the expression above missing?

Thank you!

• How does W depend on W? Also, are you trying to take a total derivative? – nbro Nov 26 '19 at 4:02
• @nbro Because W is W itself, $\frac{\partial W}{\partial W} = 1$. I am not trying to do a total derivative, but only partial derivative wrt W. – username123 Nov 26 '19 at 15:30
• can you provide a reference for the first version of $\frac{\partial h_t}{\partial W}$ – shimao Nov 30 '19 at 21:55
• @shimao Sure, one example is kharshit.github.io/blog/2019/02/22/backpropagation-through-time, where they denoted "h" as "s". – username123 Dec 2 '19 at 17:19
• @username123 in that link, they correctly write $\frac{\partial h_t}{W} = \sum_{i<t} \frac{h_t}{h_i} \frac{\partial h_i}{W}$ – shimao Dec 6 '19 at 5:35