Some web pages mentioned that LSTM causes the vanishing or exploding gradients if the sequence is too long.

These are one of the pages mention the problem:

However, I couldn't find any paper or formulation for it.
Could you please tell me the references for this problem?

  • $\begingroup$ I have not seen any, but it will be good to have a visualization/paper. A hobby project maybe $\endgroup$ – Anant Gupta Jul 17 at 9:31
  • $\begingroup$ The whole point of LSTMs is avoiding the vanishing gradient problem. $\endgroup$ – Jan Kukacka Jul 17 at 10:02

Sepp Hochreiter first describes the problem in the 1991 paper The Vanishing Gradient Problem for Recurrent Nets and Solutions. The LSTM is later proposed in the 1997 Long Short-Term Memory by Hochreiter and Schmidhuber.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.