Can Latent Dirichlet Allocation (LDA) be used to generate word embeddings?

Question

In the original Word2Vec paper (Efficient Estimation of Word Representations in Vector Space, Mikolov et al. 2013), I came across this phrase:

Many different types of models were proposed for estimating continuous representations of words, including the well-known Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA).

From this, one understands that LDA could also be used for generating dense vector representations for words, aka word embeddings, similar with what the methods proposed in this paper do (but worse).

To my very limited knowledge and understanding of LDA, this is used for topic analysis of sets of documents, and one could immediately see it as a way of maybe representing documents as vectors of topics or something similar.

But how could it be used for creating word embeddings?

Answer 1

With LDA, documents are represented as bags of words. Each word contributes to a distribution over topics for the document which you can treat as a sort of document embedding. The contribution of the single words (or topic distribution for a single-word document) can be interpreted as a word embedding. However, I am not aware of anyone using LDA in this way.