I am trying to understand the working of LSTM by considering an application. Given below is the application of LSTM for spam prediction a well known dataset.
K.clear_session()
model = Sequential()
model.add(Embedding(1000, 128, input_length=80))
model.add(LSTM(64))
model.add(Dense(256,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
Embedding layer outputs maximum 80words for each mail with each word represented as a vector with dimension of 128.
I understood the concept of unrolling the LSTM cells as it work through the sequence of words from first to 80th for each observation. Now assume that the word 'lottery' is a good predictor and this word can occur at any place in the mail (it could be first or 80th). How does LSTM learns the importance of this word? Will it keep this (importance of 'lottery') in the long-term memory across observations? If yes, how?
