1
$\begingroup$

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$. In standard PCA as far as I know we can derive $\mathbf{S}$ and $\mathbf{U}$ via two eigenvalue decompositions, of the Gram and Covariance/Correlation matrices: $$ \begin{array}{c} X=U\Sigma V^T\\ C=\dfrac{X^TX}{N-1}\\ G=\dfrac{XX^T}{N-1}\\ C=VE_CV^T\\ G=UE_GU^T\\ S=\sqrt{E_C(N-1)}\\ K=U_KE_KU^T\\ ?=VE_?V^T \end{array} $$ But how does one get $\mathbf{V}$ in the case of a kernel? All posts I've ever read only discuss $\mathbf{U}$

Note: I've read that $\mathbf{XV}=\mathbf{U\Sigma}$, however this relationship doesn't seem to hold for numpy.linalg.svd or scipy.linalg.svd

Improve this question
$\endgroup$
5
  • $\begingroup$ The nature of your question is unclear. After all, once you have obtained $U$ and $\Sigma$ from $X$, $V$ is simply given by $V = X^\prime U^\prime \Sigma^{-1}.$ $\endgroup$
    – whuber ♦
    Commented Aug 8, 2022 at 15:24
  • $\begingroup$ Where $X^\prime$ is the transpose ? Another way of phrasing what I'm asking is how does one get the principal axes/directions in Kernel PCA? $\endgroup$
    – arod
    Commented Aug 9, 2022 at 6:36
  • $\begingroup$ This would answer the question, however in SVD the $V$ matrix is $M\times M$ for an $N\times M$ input matrix $X$. The above should result in a $V$ with wrong dimension ($M\times N$). Something seems off $\endgroup$
    – arod
    Commented Aug 9, 2022 at 17:19
  • $\begingroup$ Sorry, I mixed up some transposes. But you get the point: because $U$ and $V$ are orthogonal and $\Sigma$ is diagonal, you don't have to invert any matrices and you can recover $V$ from $X,$ $U,$ and $\Sigma.$ Thus, from $X=U\Sigma V^\prime,$ you obtain $V\Sigma U^\prime = X^\prime$ via transposition and thence $V = X^\prime U \Sigma^{-}.$ $\endgroup$
    – whuber ♦
    Commented Aug 9, 2022 at 18:36
  • $\begingroup$ @whuber This is the answer I was looking for, though I can't seem to flag it as such. I'd also note (from wikipedia) $\Sigma$ is rectangular, not square as I thought $N \times M$ so this expression is valid $\endgroup$
    – arod
    Commented Aug 9, 2022 at 19:27

1 Answer 1

Reset to default
0
$\begingroup$

Based on @whuber's answer it's simply: $$ \mathbf{V}=\mathbf{X^TU\Sigma^{-1}} $$

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Aug 14, 2022 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.