1
$\begingroup$

I am new to topic modelling, so kindly bear me if my question is silly.

I am trying to calculate perplexity after applying LSA. i am aware that LSA returns negative values, so i followed the steps stated in coccaro to find the propability of each word in the vocabulary(which i am still not clear whether must i calculate for the whole vocabulary or word by word in the given test document).

Now how do i calculate perplexity?

How must i find the document propability ? must i find the sum of propabilities of individual words in the document and call it as document propability. what if the words are repeated.

update

I found the following stackoverflow question which talks about PLSA.From that , perplexity is given as 2^−∑(i=1 to N)(1/N)log2q(xi)

How can i estimate log q(xi) in that equation for LSA.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.