How do we update the parameters (weights) in recurrent neural networks?

Question

In RNNs, how do we update the weights?

I have following understanding of RNNs:

1. Parameters are shared across all time-steps, i.e.,

$$S_t = \tanh(U X_t + W S_{t-1} )$$ $$Y_k = \text{softmax}(V S_t)$$

Here $W$, $U$ and $V$ are weights. $S_t$ and $S_{t-1}$ is passage of information at time steps $t$ and $t-1$ respectively.

2. Compute loss function by summing over ALL time steps.

3. compute gradient of this loss function(which is summation over all time steps) wrt to $W$, $U$ and $V$.

Now my question is when we have these gradients (over $W$, $U$ and $V$),

1. do we update Weights and then re-compute $Y_k$ at every time step and steps 2 and 3 are repeated
2. or re-computation of $Y_k$ is done at only certain time steps?

Weights updated by gradient descent method.

I have followed following link for rnn:

http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/

Answer 1

The typical way of training RNNs (and all NNs) is that you only run forward and backward pass on the training data batch only once, i.e.,

1. Do a forward pass and compute the loss;
2. Do a backward pass and compute the loss derivate w.r.t. all trainable parameters.

The deep learning frameworks can even accumulate the gradients over multiple batches (i.e., it remembers the gradients even if the input data is no longer loaded on the GPU). The same way the gradient from RNN time steps is accumulated.

The optimizer step is done on the accumulated gradients.