I'm well aware of word embeddings (word2vec or Glove) and I know of four papers treating the subject of more general embeddings :
Distributed Representations of Sentences and Documents - Quoc V. Le, Tomas Mikolov https://arxiv.org/abs/1405.4053
Document Embedding with Paragraph Vectors - Andrew M. Dai, Christopher Olah Quoc V. Le https://arxiv.org/abs/1507.07998
An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation - Jey Han Lau, Timothy Baldwin https://arxiv.org/abs/1607.05368
which all talk about the same method and
Skip-Thought Vectors - Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler https://arxiv.org/abs/1506.06726
which maps sentences to their embeddings.
I also know that you can just take the average of the word embeddings but I am wondering two things :
- Whether it exists other ways to use word embeddings to make sentence/paragraph/document embeddings.
- Whether it exists ways of computing such embeddings without using word embeddings.
In other words, is something like sentence2vec/paragraph2vec/doc2vec possible except with the techniques in these four papers and the simple averaging process (and still obtaining good results) ?.