from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM

net = Sequential()
net.add(LSTM(units = 3, return_sequences = True, input_shape = (X_train.shape[1], 1)))
net.add(LSTM(units = 3))
net.add(Dense(units = 1))

Below image is an attempt to visually represent the above code snippet.

enter image description here

From the above figure, I understand that there are two layers of LSTM units. Each of those layers has three LSTM units/neurons. There is a single output neuron/unit. The same input is fed to all three LSTM units in the first layer.

My Questions:

a) Does the figure interpret the code sample correctly?

b) Have I described/understood the figure correct?

Thank you!


Your figure is rather accurate, yet missed one important step. In the first LSTM layer you pass a parameter return_sequences = True which passed the whole sequence of first layer to the second layer. In contrast, Dense layer receives only the last output of second LSTM layer.
I've sketched a picture of your code given a sequence on 4 input elements ($x_1, x_2, x_3, x_4$).enter image description here Each row represents a single LSTM layer. Each LSTM block contains 3 units/neurons. Within the block neurons are connected and can exchange information between sequence steps.

I would recommend this blog post to understand LSTM more.

  • $\begingroup$ Please correct me if I'm wrong, from your figure I understand that the LSTM unit has been unrolled to accommodate the input sequences(x1 x2 x3 x4). $\endgroup$ – Foshiba Sep 10 '18 at 13:02
  • $\begingroup$ Yes, that is correct. $\endgroup$ – Tautvydas Sep 10 '18 at 13:02
  • $\begingroup$ I have read Colah's blog post on LSTM. I kind of have a fair understanding of how LSTM works. But, I'm struggling to understand how it looks and functions as a whole neural network. [Q] What do you mean by "Each LSTM block contains 3 units/neurons" ?? Doesn't the LSTM block represent a neuron? $\endgroup$ – Foshiba Sep 10 '18 at 13:11
  • $\begingroup$ LSTM block would be a a cube on the graph. Typically LSTM block has more than one neuron inside. Say in your example on 3 neurons, when you are computing, say, forget gate $f_t$ then all hidden state variables $h_(t-1)$ are considered for each neuron. Does that make sense? $\endgroup$ – Tautvydas Sep 10 '18 at 13:27
  • $\begingroup$ Can I say that each neuron will have all the gates of the LSTM? $\endgroup$ – Foshiba Sep 10 '18 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.