I have a rather basic question about Probabilistic Principal Component Analysis, which I am now trying to apply to a real-world problem.
In PPCA, the crucial assumption is that the generating process of the observations in $R^n$ is $t=Wx +\sigma^2\epsilon$, where x are iid standard gaussians in $R^q$ (with $q \le n$) and $\epsilon$ are iid standard gaussian vectors in $R^n$. The authors find that the MLE solution is $\hat W=U_q (\Lambda_q - \sigma^2 I)^{1/2}$, where $\Lambda_q$ is the diagonal matrix of the $q$ largest eigenvalues of the empirical covariance matrix, and $U_q$ is the corresponding eigenvectors submatrix, and $\sigma^2$ is the average of the remaining smaller eigenvalues (see sec. 3.2 of the linked paper).
My question is simple. The covariance matrix is $C=WW'+\sigma^2 I$. Directly plugging in the above result, we obtain $\hat C=U_q\Lambda_q U_q$ (the $\sigma^2$ terms cancel out). Can this be correct? The covariance matrix would be rank-deficient. Am I missing something?
