# PCA eigenvectors with dimensionality reduction

I want to understand how I can compute the eigenvectors and the eigenvalues of a matrix using dimensional reduction.I have a Matrix $M$ of dimensions $n$ x $d$ using dimension reduction I can compute the eigenvectors and the eigenvalues of the Covariance matrix $MM^t$. After computing these eigenvectors and eigenvalues how can I compute the eigenvectors of the original matrix?

• Note that the original matrix does not have to be square but the covariance matrix is necessarily square. In non-square matrices do not have eigenvalues (they do have singular values though). – Bitwise Nov 4 '12 at 18:54
• Ok but I cant understand after finding the eigenvectors and eigenvalues for the Covariance matrix how to compute the eigenvectors for the original? – nikosdi Nov 4 '12 at 18:58
• are N and D normalizing/centering matrices? – Bitwise Nov 4 '12 at 19:11
• The matrix M with Dimensions NxD is centered.I calculated the eigenvalues and eigenvectors of MM'/N and I want the eigenvector of the original.Maybe this is a bit wrong?The eigenvector of the original will have dimension D – nikosdi Nov 4 '12 at 19:13
• Ok, please change it so it won't be capitals, because this is misleading. What I meant in my comment is that your original matrix will have eigenvalues only if $n=d$ (i.e. it is square). – Bitwise Nov 4 '12 at 19:26

## 1 Answer

You original matrix is rectangular, for which eigenvalues and eigenvectors are not defined. However, there are similar objects called singular values and singular vectors which are defined. They are related as follows:

• The left-singular vectors of $M$ are eigenvectors of $MM^t$.
• The right-singular vectors of $M$ are eigenvectors of $M^tM$.
• The non-zero-singular values of M are the square roots of the non-zero eigenvalues of both $M^tM$ and $MM^t$.

So if your original matrix $M$ was square, the eigenvalues of $M$ are just the squares of the eigenvalues of $MM^t$ which you calculated.

You can find more details in Wikipedia's article on Singular Value Decomposition (SVD), which is closely related to PCA.

• My question was a bit wrong i guess but I wont delete it.I was trying to find the eigen decomposition of the matrix M'M using the matrix MM'.After I have calculated the eigenvectors of MM' the eigen vectors of M'M was e*M and the eigenvalues was the same. – nikosdi Nov 6 '12 at 16:08
• @nikosdi you can post a new question. – Bitwise Nov 6 '12 at 16:10
• Maybe it is better to edit this ? – nikosdi Nov 6 '12 at 16:11
• @nikosdi since both the question and answer are different from what you are asking for, I think a new question would be better. Both forms of the question could be useful for people. – Bitwise Nov 6 '12 at 16:12
• Ok thank you! Your answer helped me a lot understanding what I do.I will create a new question. – nikosdi Nov 6 '12 at 16:14