I want to understand how I can compute the eigenvectors and the eigenvalues of a matrix using dimensional reduction.I have a Matrix $M$ of dimensions $n$ x $d$ using dimension reduction I can compute the eigenvectors and the eigenvalues of the Covariance matrix $MM^t$. After computing these eigenvectors and eigenvalues how can I compute the eigenvectors of the original matrix?
$\begingroup$
$\endgroup$
7
-
1$\begingroup$ Note that the original matrix does not have to be square but the covariance matrix is necessarily square. In non-square matrices do not have eigenvalues (they do have singular values though). $\endgroup$– BitwiseNov 4, 2012 at 18:54
-
$\begingroup$ Ok but I cant understand after finding the eigenvectors and eigenvalues for the Covariance matrix how to compute the eigenvectors for the original? $\endgroup$– nikosdiNov 4, 2012 at 18:58
-
$\begingroup$ are N and D normalizing/centering matrices? $\endgroup$– BitwiseNov 4, 2012 at 19:11
-
$\begingroup$ The matrix M with Dimensions NxD is centered.I calculated the eigenvalues and eigenvectors of MM'/N and I want the eigenvector of the original.Maybe this is a bit wrong?The eigenvector of the original will have dimension D $\endgroup$– nikosdiNov 4, 2012 at 19:13
-
$\begingroup$ Ok, please change it so it won't be capitals, because this is misleading. What I meant in my comment is that your original matrix will have eigenvalues only if $n=d$ (i.e. it is square). $\endgroup$– BitwiseNov 4, 2012 at 19:26
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
5
You original matrix is rectangular, for which eigenvalues and eigenvectors are not defined. However, there are similar objects called singular values and singular vectors which are defined. They are related as follows:
- The left-singular vectors of $M$ are eigenvectors of $MM^t$.
- The right-singular vectors of $M$ are eigenvectors of $M^tM$.
- The non-zero-singular values of M are the square roots of the non-zero eigenvalues of both $M^tM$ and $MM^t$.
So if your original matrix $M$ was square, the eigenvalues of $M$ are just the squares of the eigenvalues of $MM^t$ which you calculated.
You can find more details in Wikipedia's article on Singular Value Decomposition (SVD), which is closely related to PCA.
-
$\begingroup$ My question was a bit wrong i guess but I wont delete it.I was trying to find the eigen decomposition of the matrix M'M using the matrix MM'.After I have calculated the eigenvectors of MM' the eigen vectors of M'M was e*M and the eigenvalues was the same. $\endgroup$– nikosdiNov 6, 2012 at 16:08
-
-
-
$\begingroup$ @nikosdi since both the question and answer are different from what you are asking for, I think a new question would be better. Both forms of the question could be useful for people. $\endgroup$– BitwiseNov 6, 2012 at 16:12
-
$\begingroup$ Ok thank you! Your answer helped me a lot understanding what I do.I will create a new question. $\endgroup$– nikosdiNov 6, 2012 at 16:14