10
$\begingroup$

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix?

What happens when the covariance matrix is not diagonalizable, i.e. its eigenvectors do not span the complete vector space?

$\endgroup$

1 Answer 1

10
$\begingroup$

Covariance matrix is a symmetric matrix, hence it is always diagonalizable.

In fact, in the diagonalization, $C=PDP^{-1}$, we know that we can choose $P$ to be an orthogonal matrix.

It belongs to a larger class of matrix known as Hermitian matrix that guarantees that they can be diagonalized.

$\endgroup$
2
  • $\begingroup$ what if there are repeated eigenvalues? how do you ensure the orthogonality then? $\endgroup$ Commented Sep 24, 2023 at 15:40
  • 1
    $\begingroup$ It doesn't require the eigenvalues to be distinct. One possible way to prove it is via Schur's theorem. I was introduced to the theorem via mathematical induction many years ago. $\endgroup$ Commented Sep 24, 2023 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.