I'm reading up on word embeddings and am a bit confused. It seems there are a couple approaches:
1) Use an unsupervised approach to generate word embeddings (basically predicting the probability of a word A being in the neighborhood of a word B). Then you take the weights matrix and each row corresponds to a word embedding.
2) Add an embedding layer at the start of your neural network. It takes in a one-hot encoded document (suppose) and this weights matrix learns word embeddings that are useful for the task at hand. Maybe classifying documents by author uses word embeddings that are pretty different from approach #1.
Here's my confusion: In approach #1, I can maintain the order of words in my document by just going through and replacing each word by its word embedding.
How does approach #2 allow me to do this? I'd like to learn the word embeddings at train time but also take advantage of the order in which the words occur. If I simply one-hot encode first, I lose out on the order-based information.
Any ideas how I can both learn embeddings during training and take advantage of the words' ordering?
Thanks!