Recently Julia Silge blogged here and here, quoting blog entry by Chris Moody, who suggested that the similarities arithmetic in word2vec can be approximated by using PMI indexes followed by SVD decomposition. Word2vec is used to produce word vectors, that enables a user to do similarities arithmetics, e.g. king - man + woman = queen. As suggested in the blog entries, word2vec can be easily approximated using PMI indexes, i.e. $\log \tfrac{p(x,\,y)}{p(x)\,p(y)}$, and then decomposing it using SVD.

My questions are, why exactly did they choose such combination of methods to approximate word2vec and why exactly does it work like intended? I understand why SDV can be used for recommender systems etc., but I am curious with such paring it with PMI.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.