Suppose I have some noisy dataset $\mathbf{X} \in \mathbb{R}^{N \times p}$ that I want to perform PCA on. Obtaining the (trimmed) SVD $\mathbf{X} = \mathbf{UDV}$ infers the q-dimensional principal components $\mathbf{Z} = \mathbf{U}_q \mathbf{D}_q \in \mathbb{R}^{N \times q}$.

But suppose I have some side-information for predicting these principal components, which gives me q-dimensional predicted-PCs $\mathbf{C} \in \mathbb{R}^{N \times q}$ for each of the $N$ samples. This suggests I might want to adjust the PCA objective to add a term that minimizes $\|\mathbf{U}_q \mathbf{D}_q - \mathbf{C} \|^2$.

Are there existing papers and algorithms that do this? And if not, is this because it would be really difficult to solve the optimization problem with that additional term?

  • $\begingroup$ Is this teh same problem as discussed in the following thread? stats.stackexchange.com/questions/473267/… $\endgroup$
    – cdalitz
    Dec 17, 2020 at 13:52
  • $\begingroup$ @cdalitz See stats.stackexchange.com/questions/473267/… for the reason I edited your comment. $\endgroup$
    – whuber
    Dec 17, 2020 at 15:38
  • $\begingroup$ No, it is not the same problem. If you fix the principal components to some given vectors, it is trivial as shown in the answer to that question. But if you add another term, you end up with a more complicated loss function. Also, presumably in that question the orthonormality constraint isn't applicable, while here it still is. $\endgroup$ Dec 17, 2020 at 16:37


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.