# Recurrent neural networks - why is the vanishing gradient a problem?

I understand why the gradient tends to become small in the early layers of a deep network. However I am trying to clarify why this is a problem in the case of RNNs.

This is what I understand: the gradient $\frac{\partial E}{\partial w}$ of a weight $w$ is computed as the average of the gradients at each time step. If the gradients of the early time steps are close to $0$, then they have no influence on the final gradient. Effectively this means the beginning of the sequence has no influence on how the RNN weights are learnt.

Is this the problematic aspect of the vanishing gradient?