Timeline for Why cannot I obtain a valid SVD of X via eigenvalue decomposition of XX' and X'X?
Current License: CC BY-SA 3.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 27, 2020 at 16:37 | comment | added | Single Malt | Yes that is what I meant, I had forgotten it was $XX’$—the concept is clearer after reading the link. So the set of eigenvalues of $XX’$ correspond to the squares of the singular values. | |
| Nov 27, 2020 at 15:56 | comment | added | whuber♦ | @Single Presuming you mean eigenvalues of $X^\prime X$ or $XX^\prime,$ then the squares of the singular values of $X$ equal those eigenvalues. See stats.stackexchange.com/a/220324/919 for a detailed account of these relationships. | |
| Nov 27, 2020 at 11:45 | comment | added | Single Malt | I could have communicated that last sentence better, as it is imprecise. When performing PCA using SVD, the SVD matrix $D$ diagonal values, do they have any relationship to the eigenvalues? | |
| Nov 26, 2020 at 23:23 | comment | added | whuber♦ | @Single Everything you write sounds good, but I'm unsure what you mean by "decreasing order diagonals." | |
| Nov 26, 2020 at 17:21 | comment | added | Single Malt | Got it, its an optional way of outputting the SVD, de facto rather than necessary, but does have the advantage of being consistent across implementations and also may be a result of LAPACK commonality between software implementations. For PCA, are the decreasing order diagonals “paired” with the decreasing order eigenvalues, or are they unrelated apart from both being ordered? | |
| Nov 26, 2020 at 16:37 | comment | added | whuber♦ | @Single One aspect of PCA is to order the principal components by their eigenvalues (descending), whence order does matter. Most (if not all) SVD implementations order the eigenvalues, even though this is not a mathematical requirement of SVD. | |
| Nov 26, 2020 at 14:28 | comment | added | Single Malt | Slightly unrelated, when SVD is used for a PCA calculation, is the SVD used such that the diagonal entries of $\Sigma$ are monotonic in their matrix $ii$ index? Or reworded, SVD is a generalization of eigendecomposition, but I am confused about how this is done and which of the non-unique SVDs is used--or does it not matter? | |
| Feb 8, 2017 at 17:37 | comment | added | Federico Poloni | You are correct, this is only an edge case, and a tricky one indeed. In some sense, it is another manifestation of the same problem that you outline in your answer, that this method doesn't ensure a "matching" between the columns of $U$ and $V$. Computing the SVD starting from the eigendecompositions is still a great learning example. | |
| Feb 8, 2017 at 16:41 | comment | added | whuber♦ | @Federico Thank you for that reminder. You are quite correct--I have implicitly assumed all eigenvalues are distinct, for indeed that is almost surely going to be the case in statistical applications and one gets out of the habit of considering the ambiguities with "degenerate" eigenspaces. | |
| Feb 8, 2017 at 16:27 | comment | added | Federico Poloni |
Unfortunately, this method doesn't work for every matrix. The problem is that there may be repeated eigenvalues, so the eigendecomposition of $A'A$ and $AA'$ is not unique and not all choices of $U$ and $V$ can be used to retrieve the SVD. For instance, if you take any non-diagonal orthogonal matrix (say, $A=\begin{bmatrix}3/5&4/5\\-4/5&3/5\end{bmatrix}$), then $AA'=A'A=I$ and hence eigen will return $U=V=I$, thus $U'AV=A$ is not diagonal.
|
|
| Feb 4, 2017 at 16:06 | comment | added | processing_statistician | Thank you for your kind help! I think I understand this issue (finally). | |
| Feb 4, 2017 at 0:29 | vote | accept | processing_statistician | ||
| Feb 3, 2017 at 23:53 | history | edited | whuber♦ | CC BY-SA 3.0 |
added 666 characters in body
|
| Feb 3, 2017 at 23:38 | comment | added | whuber♦ | @Amoeba That is right: in the spirit of hand-computing an SVD, which evidently is an educational exercise, no attention is paid here to efficiency. | |
| Feb 3, 2017 at 23:36 | comment | added | amoeba |
+1. This is very clear. I would only add that in practice it's enough to compute either U or V and then to obtain another matrix via multiplying with A. This way one performs only one (instead of two) eigendecompositions, and the signs will come out right.
|
|
| Feb 3, 2017 at 23:32 | history | answered | whuber♦ | CC BY-SA 3.0 |