Understanding RNN/LSTM I am trying to understand how to use RNN/LSTM; but I cannot understand some concepts related to topology. I found really useful information in this article:
http://colah.github.io/posts/2015-08-Understanding-LSTMs/. But then, when comparing that topology 1 to the one Alex Graves says in his LSTM writing2, and the one described here 3, we can see the differences (in the first one the inputs never go to previous states, whereas they do in the other two topologies).
What is going on?
Thank you
 A: There exist several LSTM variants, this may explain the difference you have observed.
{1} gives a list:

4.3. LSTM Variants
The vanilla LSTM from Section 2 is referred as Vanilla (V). The
  derived eight variants of the V architecture are the following:
  
  
*
  
*No Input Gate (NIG)
  
*No Forget Gate (NFG)
  
*No Output Gate (NOG)
  
*No Input Activation Function (NIAF)
  
*No Output Activation Function (NOAF)
  
*No Peepholes (NP)
  
*Coupled Input and Forget Gate (CIFG)
  
*Full Gate Recurrence (FGR)
  
  
  The first six variants are self-explanatory. The CIFG variant uses
  only one gate for gating both the input and the cell recurrent
  self-connection (an LSTM modification proposed in GRU (Cho et al.,
  2014)). This is equivalent to setting $f_t = 1 − i_t$ instead of
  learning the forget gate weights independently. The FGR variant adds
  recurrent connections between all the gates as in the original
  formulation of the LSTM (Hochreiter & Schmidhuber, 1997). It adds nine
  additional recurrent weight matrices, thus significantly increasing
  the number of parameters.


References:


*

*{1} Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, Jürgen Schmidhuber
LSTM: A Search Space Odyssey. https://arxiv.org/abs/1503.04069
