I have a dataset. Each data point of this set contains a variable length of sequences with 7 letters. For example, Data point 1 has a sequence (A, A, B, E, B, C, D, E, D.....). I used LSTM to predict the next letter from this dataset. I know that adding Attention mechanism can improve the prediction accuracy. I searched for the background of how actually attention mechanism works. I came across various models which work well on the encoder-decoder problem. I am wondering in my case (i.e., predicting the next action/word) how Attention mechanism can improve the accuracy. If anyone can give some intuition or reference, it will be a great benefit for me.