I have a question about covariance matrices. I have read one interesting property that, all symmetric matrices are diagonalizable.

Suppose we have a data matrix $X$ that has only $m$ independent columns. Additionally, the corresponding covariance matrix $$\Sigma = (X - \mu)^T(X -\mu)$$

My question is, say the covariance matrix, $\Sigma$, lives in $R^{p *p}$, but this matrix will only have $m$ independent columns. Does this imply that we still get $p$ (full) orthogonal eigenvectors or only $m$ orthogonal eigenvectors and the remaining $p - m$ independent but not orthogonal eigenvectors?

  • 1
    $\begingroup$ For a symmetric matrix with $p$ rows and columns, you can always find $p$ orthogonal eigenvalues see here. $\endgroup$ – John L Mar 13 at 14:22
  • 1
    $\begingroup$ For some intuition, consider the case $m=0.$ $\endgroup$ – whuber Mar 13 at 18:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.