# Clarifications regarding LSTM

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM

net = Sequential()
net.add(LSTM(units = 3, return_sequences = True, input_shape = (X_train.shape[1], 1)))
net.add(LSTM(units = 3))
net.add(Dense(units = 1))


Below image is an attempt to visually represent the above code snippet.

From the above figure, I understand that there are two layers of LSTM units. Each of those layers has three LSTM units/neurons. There is a single output neuron/unit. The same input is fed to all three LSTM units in the first layer.

My Questions:

a) Does the figure interpret the code sample correctly?

b) Have I described/understood the figure correct?

Thank you!

## 1 Answer

Your figure is rather accurate, yet missed one important step. In the first LSTM layer you pass a parameter return_sequences = True which passed the whole sequence of first layer to the second layer. In contrast, Dense layer receives only the last output of second LSTM layer.
I've sketched a picture of your code given a sequence on 4 input elements ($x_1, x_2, x_3, x_4$). Each row represents a single LSTM layer. Each LSTM block contains 3 units/neurons. Within the block neurons are connected and can exchange information between sequence steps.

I would recommend this blog post to understand LSTM more.

• Please correct me if I'm wrong, from your figure I understand that the LSTM unit has been unrolled to accommodate the input sequences(x1 x2 x3 x4). Sep 10 '18 at 13:02
• Yes, that is correct. Sep 10 '18 at 13:02
• I have read Colah's blog post on LSTM. I kind of have a fair understanding of how LSTM works. But, I'm struggling to understand how it looks and functions as a whole neural network. [Q] What do you mean by "Each LSTM block contains 3 units/neurons" ?? Doesn't the LSTM block represent a neuron? Sep 10 '18 at 13:11
• LSTM block would be a a cube on the graph. Typically LSTM block has more than one neuron inside. Say in your example on 3 neurons, when you are computing, say, forget gate $f_t$ then all hidden state variables $h_(t-1)$ are considered for each neuron. Does that make sense? Sep 10 '18 at 13:27
• Can I say that each neuron will have all the gates of the LSTM? Sep 10 '18 at 13:37