# How to derive the three matrices of SVD from eigenvalue decomposition in Kernel PCA?

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $$\mathbf{K}$$ and standard PCA via SVD of the input $$\mathbf{X}$$. In standard PCA as far as I know we can derive $$\mathbf{S}$$ and $$\mathbf{U}$$ via two eigenvalue decompositions, of the Gram and Covariance/Correlation matrices: $$\begin{array}{c} X=U\Sigma V^T\\ C=\dfrac{X^TX}{N-1}\\ G=\dfrac{XX^T}{N-1}\\ C=VE_CV^T\\ G=UE_GU^T\\ S=\sqrt{E_C(N-1)}\\ K=U_KE_KU^T\\ ?=VE_?V^T \end{array}$$ But how does one get $$\mathbf{V}$$ in the case of a kernel? All posts I've ever read only discuss $$\mathbf{U}$$

Note: I've read that $$\mathbf{XV}=\mathbf{U\Sigma}$$, however this relationship doesn't seem to hold for numpy.linalg.svd or scipy.linalg.svd

• The nature of your question is unclear. After all, once you have obtained $U$ and $\Sigma$ from $X$, $V$ is simply given by $V = X^\prime U^\prime \Sigma^{-1}.$
– whuber
Commented Aug 8, 2022 at 15:24
• Where $X^\prime$ is the transpose ? Another way of phrasing what I'm asking is how does one get the principal axes/directions in Kernel PCA?
– arod
Commented Aug 9, 2022 at 6:36
• This would answer the question, however in SVD the $V$ matrix is $M\times M$ for an $N\times M$ input matrix $X$. The above should result in a $V$ with wrong dimension ($M\times N$). Something seems off
– arod
Commented Aug 9, 2022 at 17:19
• Sorry, I mixed up some transposes. But you get the point: because $U$ and $V$ are orthogonal and $\Sigma$ is diagonal, you don't have to invert any matrices and you can recover $V$ from $X,$ $U,$ and $\Sigma.$ Thus, from $X=U\Sigma V^\prime,$ you obtain $V\Sigma U^\prime = X^\prime$ via transposition and thence $V = X^\prime U \Sigma^{-}.$
– whuber
Commented Aug 9, 2022 at 18:36
• @whuber This is the answer I was looking for, though I can't seem to flag it as such. I'd also note (from wikipedia) $\Sigma$ is rectangular, not square as I thought $N \times M$ so this expression is valid
– arod
Commented Aug 9, 2022 at 19:27

Based on @whuber's answer it's simply: $$\mathbf{V}=\mathbf{X^TU\Sigma^{-1}}$$