5
$\begingroup$

I have a rather basic question about Probabilistic Principal Component Analysis, which I am now trying to apply to a real-world problem.

In PPCA, the crucial assumption is that the generating process of the observations in $R^n$ is $t=Wx +\sigma^2\epsilon$, where x are iid standard gaussians in $R^q$ (with $q \le n$) and $\epsilon$ are iid standard gaussian vectors in $R^n$. The authors find that the MLE solution is $\hat W=U_q (\Lambda_q - \sigma^2 I)^{1/2}$, where $\Lambda_q$ is the diagonal matrix of the $q$ largest eigenvalues of the empirical covariance matrix, and $U_q$ is the corresponding eigenvectors submatrix, and $\sigma^2$ is the average of the remaining smaller eigenvalues (see sec. 3.2 of the linked paper).

My question is simple. The covariance matrix is $C=WW'+\sigma^2 I$. Directly plugging in the above result, we obtain $\hat C=U_q\Lambda_q U_q$ (the $\sigma^2$ terms cancel out). Can this be correct? The covariance matrix would be rank-deficient. Am I missing something?

$\endgroup$
6
$\begingroup$

Unless I'm missing something, I think $U_q U_q' \neq I$ here (almsot surely, at least). The columns of $U_q$ are orthogonal, not the rows since the last $n-q$ columns are removed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.