Say I have the following code to create a LSTM layer:

lstm_model = Sequential()
lstm_model.add(LSTM(128, batch_input_shape=(BATCH_SIZE, TIME_STEPS, FEATURES))

Lets say, I plan to train a model where there are n tanks. for each tank the pressure and temperature are recorded at an interval of 1 hour, a total of 5 times everyday. The data is recorded for a year. Thus I will have batch_size = 365, time steps equal to 5 and number of features = 2. So my input shape will be (365,5,2).

So will the code mentioned earlier create an entire LSTM layer with total number of cells just equal to 5?


1 Answer 1

  • The batch size is the size of the training batch you use. It can be anything and doesn't effect the size of the LSTM layer. It just modifies the gradient update step.

  • Time steps is the one that determines the size, because it's the number of times that you unroll your LSTM cell. So, that is right, total number of unrolled cells is equal to $5$.

  • The features is related to the series you want to input/predict. If it is $1$, the series is univariate, otherwise it is multi-variate and doesn't have anything to do with the number of cells unrolled.

  • 1
    $\begingroup$ What exactly is the LSTM cell? How many LSTM cells are there in my example? $\endgroup$
    – A-ar
    Feb 4, 2020 at 11:27
  • $\begingroup$ You have one cell, but it is unrolled $5$ times, so it appears as if you have $5$ due to the recursive nature.The term isn't consistent across libraries and literature by the way. $\endgroup$
    – gunes
    Feb 4, 2020 at 13:19
  • $\begingroup$ OK, So I suppose each of the unrolled cells will have two neurons? since there are two features in my example. $\endgroup$
    – A-ar
    Feb 4, 2020 at 15:41
  • $\begingroup$ The neuron concept doesn't apply well to RNNs because in feedforward nets neurons have scalar outputs. However, LSTM cell outpus the hidden state, $h_t$, which is 128 in your case. So, it's as if there are 128 neurons in the cell producing outputs. In general, the final estimation is done via a subsequent fully connected layer. I suggest you keep the two concepts separate. $\endgroup$
    – gunes
    Feb 4, 2020 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.