I try to create a sentiment analysis that have 7 classification. Let's say, I have 100.000 unique word (already converted into 100.000 integer) which have the longest input is 41. I created 3 layer embedding, LSTM and Dense Layer.
model = Sequential(); model.add(Embedding(100000, 50, input_length = 41)) model.add(LSTM(75, dropout=0.2, recurrent_dropout=0.0)) model.add(Dense(7, activation='softmax')) optimizer = Adam(lr=0.00001) model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
Based on code above, I tried to create the illustration model as below :
From my understanding, Xt is 41 integer that will be an input. The embedding layer will convert every integer into real-valued vector of length 50. Every vector that has been converted will be an input of LSTM layer (X1 - X41)
- How many LSTM units created based on the code (as my understanding is 41) ?
Because of return_sequences & return_state by default false for LSTM layer, so it will become many to one ?
If so, h41 is the only output value that will be used for classification ?