I don't understand how adding the cosine encodings/functions to each of the dimension of the word vector embedding enables the network to "understand" where each word is situated in the sentence.
What is the intuition behind it? It seems a bit counter intuitive to me to just add these values to the word embedding, they are 2 very different things.
Is the reasoning that it does not make sense for a single example but adding the same values over and over for thousands/millions of input sentences will enable the network to dissociate it?
Essentially the same word at different positions in the sentence will have slightly different embedding and this is where the network is able to capture the position information? It seems to me that it would be more intuitive to concatenate the cosine embedding rather than adding it.
Thanks a lot