1
$\begingroup$

In the original Word2Vec paper (Efficient Estimation of Word Representations in Vector Space, Mikolov et al. 2013), I came across this phrase:

Many different types of models were proposed for estimating continuous representations of words, including the well-known Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA).

From this, one understands that LDA could also be used for generating dense vector representations for words, aka word embeddings, similar with what the methods proposed in this paper do (but worse).

To my very limited knowledge and understanding of LDA, this is used for topic analysis of sets of documents, and one could immediately see it as a way of maybe representing documents as vectors of topics or something similar.

But how could it be used for creating word embeddings?

$\endgroup$

1 Answer 1

1
$\begingroup$

With LDA, documents are represented as bags of words. Each word contributes to a distribution over topics for the document which you can treat as a sort of document embedding. The contribution of the single words (or topic distribution for a single-word document) can be interpreted as a word embedding. However, I am not aware of anyone using LDA in this way.

$\endgroup$
1
  • $\begingroup$ "The contribution of the single words (or topic distribution for a single-word document) can be interpreted as a word embedding." can make sense in a way, so I'd guess the word2vec author also came across smth like that in their reviews of the problem space and thought worth mentioning... Thanks, at least now I don't feel dumb for thinking it's smth well known I hand no idea of :) I'll take this as an answer $\endgroup$
    – NeuronQ
    May 9, 2020 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.