I'm building a custom LSTM net based on this article. I got questions on how to implement the backpropagation, based on these formulas of the derivatives in an LSTM layer:
Question 1: The weights (w.., v.., b..) in the formulas for dHt and dXt have a T above them. Does that mean we have to use the "original" weights that have been used at that time step during propagation (and not current weights which might have been updated since then)?
Question 2: When we start backpropagation through time, where do we get dCt+1 and dHt+1 from? As we go back in time, they are accumulated (as shown in the formulas), but at the first iteration, do we just set them to 0?