I am not sure how to calculate my LSTM weights based on this link and my Keras programing as below:

model = Sequential()
model.add(LSTM1(A1, input_shape=(2,B1), return_sequences=True, stateful=False)) 
model.add(LSTM2(A2, return_sequences=True, stateful=False))
model.add(LSTM3(A3, return_sequences=False, stateful=False))
model.add(Dense(B2, activation='softmax'))

Is that correct?

Input = A1
Output = B2
Hidden Layers = 2
Cells in layer number one = A2
Cells in layer number two = A3


4*ni*(ni−1+ni) for each layer is calculated as below:
Total number of weights = X1+X2+X3+X4

All my problem is about understanding the concept of cell in Keras and not in LSTM itself.

  • $\begingroup$ Does your parameter count match what you get from model.summary()? If it's different, how? $\endgroup$
    – Sycorax
    May 29 '20 at 13:30

Your arithmetic won't give you the total number of parameters fore several reasons.

First, you're subtracting 1 from the parameter count. The notation in the linked post is using a subscript $n_{i-1}$, which is an index, and does not mean $n_i - 1$.

Additionally, your arithmetic is incorrect because you're not using the input and output sizes correctly.

Finally, you're using bias neurons in your LSTM, so you need too add those in as well.

Making these corrections, the number of parameters in the first LSTM layer is X1=4*A1*(B1+A1)+4*A1, and the second is X2=4*A2*(A1+A2)+4*A2.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.