I want to understand how I can compute the eigenvectors and the eigenvalues of a matrix using dimensional reduction.I have a Matrix $M$ of dimensions $n$ x $d$ using dimension reduction I can compute the eigenvectors and the eigenvalues of the Covariance matrix $MM^t$. After computing these eigenvectors and eigenvalues how can I compute the eigenvectors of the original matrix?

  • 1
    $\begingroup$ Note that the original matrix does not have to be square but the covariance matrix is necessarily square. In non-square matrices do not have eigenvalues (they do have singular values though). $\endgroup$
    – Bitwise
    Nov 4, 2012 at 18:54
  • $\begingroup$ Ok but I cant understand after finding the eigenvectors and eigenvalues for the Covariance matrix how to compute the eigenvectors for the original? $\endgroup$
    – nikosdi
    Nov 4, 2012 at 18:58
  • $\begingroup$ are N and D normalizing/centering matrices? $\endgroup$
    – Bitwise
    Nov 4, 2012 at 19:11
  • $\begingroup$ The matrix M with Dimensions NxD is centered.I calculated the eigenvalues and eigenvectors of MM'/N and I want the eigenvector of the original.Maybe this is a bit wrong?The eigenvector of the original will have dimension D $\endgroup$
    – nikosdi
    Nov 4, 2012 at 19:13
  • $\begingroup$ Ok, please change it so it won't be capitals, because this is misleading. What I meant in my comment is that your original matrix will have eigenvalues only if $n=d$ (i.e. it is square). $\endgroup$
    – Bitwise
    Nov 4, 2012 at 19:26

1 Answer 1


You original matrix is rectangular, for which eigenvalues and eigenvectors are not defined. However, there are similar objects called singular values and singular vectors which are defined. They are related as follows:

  • The left-singular vectors of $M$ are eigenvectors of $MM^t$.
  • The right-singular vectors of $M$ are eigenvectors of $M^tM$.
  • The non-zero-singular values of M are the square roots of the non-zero eigenvalues of both $M^tM$ and $MM^t$.

So if your original matrix $M$ was square, the eigenvalues of $M$ are just the squares of the eigenvalues of $MM^t$ which you calculated.

You can find more details in Wikipedia's article on Singular Value Decomposition (SVD), which is closely related to PCA.

  • $\begingroup$ My question was a bit wrong i guess but I wont delete it.I was trying to find the eigen decomposition of the matrix M'M using the matrix MM'.After I have calculated the eigenvectors of MM' the eigen vectors of M'M was e*M and the eigenvalues was the same. $\endgroup$
    – nikosdi
    Nov 6, 2012 at 16:08
  • $\begingroup$ @nikosdi you can post a new question. $\endgroup$
    – Bitwise
    Nov 6, 2012 at 16:10
  • $\begingroup$ Maybe it is better to edit this ? $\endgroup$
    – nikosdi
    Nov 6, 2012 at 16:11
  • $\begingroup$ @nikosdi since both the question and answer are different from what you are asking for, I think a new question would be better. Both forms of the question could be useful for people. $\endgroup$
    – Bitwise
    Nov 6, 2012 at 16:12
  • $\begingroup$ Ok thank you! Your answer helped me a lot understanding what I do.I will create a new question. $\endgroup$
    – nikosdi
    Nov 6, 2012 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.