0
$\begingroup$

I try to create a sentiment analysis that have 7 classification. Let's say, I have 100.000 unique word (already converted into 100.000 integer) which have the longest input is 41. I created 3 layer embedding, LSTM and Dense Layer.

model = Sequential(); 
model.add(Embedding(100000, 50, input_length = 41))
model.add(LSTM(75, dropout=0.2, recurrent_dropout=0.0)) 
model.add(Dense(7, activation='softmax'))
optimizer = Adam(lr=0.00001)
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])

Based on code above, I tried to create the illustration model as below :

enter image description here

From my understanding, Xt is 41 integer that will be an input. The embedding layer will convert every integer into real-valued vector of length 50. Every vector that has been converted will be an input of LSTM layer (X1 - X41)

  1. How many LSTM units created based on the code (as my understanding is 41) ?
    Because of return_sequences & return_state by default false for LSTM layer, so it will become many to one ?
    If so, h41 is the only output value that will be used for classification ?
$\endgroup$
2
$\begingroup$

Interestingly I searched for this answer for a long time and then I got an answer. It is actually one LSTM unit that is repeated by the length of the sentence or collection of records. This was excellently explained here

https://zhuanlan.zhihu.com/p/58854907'

64 is the number of dense units inside - Isn't that interesting?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.