In the original paper of pLSA the author, Thomas Hoffman, draw a parallel between pLSA and LSA data structures that I would like to discuss with you.
Background:
Taking inspiration the Information Retrieval suppose we have a collection of $N$ documents $$D = \lbrace d_1, d_2, ...., d_N \rbrace$$ and a vocabulary of $M$ terms $$\Omega = \lbrace \omega_1, \omega_2, ..., \omega_M \rbrace$$
A corpus $X$ can be represented by a $N \times M$ matrix of cooccurences.
In the Latent Semantic Analisys by SVD the matrix $X$ is factorized in three matrices: $$X = U \Sigma V^T$$ where $\Sigma = diag \lbrace \sigma_1, ..., \sigma_s \rbrace$ and the $\sigma_i$ are the singular values of $X$ and $s$ is the rank of $X$.
The LSA approximation of $X$ $$\hat{X} = \hat{U}\hat{\Sigma}\hat{V^T}$$is then computed truncating the three matrices to some level $k < s$, as shown in the picture:
In pLSA, choosen a fixed set of topics (latent variables) $Z = \lbrace z_1, z_2, ..., z_Z \rbrace$ the approximation of $X$ is computed as: $$X = [P(d_i | z_k)] \times [diag(P(z_k)] \times [P(f_j|z_k)]^T $$ where the three matrices are the ones that maximize the likelihood of the model.
Actual question:
The author states that these relations subsist:
- $U = [P(d_i | z_k)]$
- $\hat{\Sigma} = [diag(P(z_k)]$
- $V = [P(f_j|z_k)]$
and that the crucial difference between LSA and pLSA is the objective function utilized to determine the optimal decomposition/approximation.
I'm not sure he is right, since I think that the two matrices $\hat{X}$ represemt different concepts: in LSA it is an approximation of the number of time a term appears in a document, and in pLSA is the (estimated) probability that a term appear in the document.
Can you help me clarify this point?
Furthermore, suppose we have computed the two models on a corpus, given a new document $d^*$, in LSA I use to compute it approximation as: $$\hat{d^*} = d^*\times V \times V^T$$
- Is this always valid?
- Why I do not get meaningful result applying the same procedure to pLSA? $$\hat{d^*} = d^*\times [P(f_j|z_k)] \times [P(f_j|z_k)]^T$$
Thank you.