In PCA, why do we assume that the covariance matrix is always diagonalizable?

Question

In Principal Component Analysis, what is the justification for the assumption that the covariance matrix is always a diagonalizable matrix?

What happens when the covariance matrix is not diagonalizable, i.e. its eigenvectors do not span the complete vector space?

Siong Thye Goh · Accepted Answer · 2018-02-16 08:57:16Z

9

Covariance matrix is a symmetric matrix, hence it is always diagonalizable.

In fact, in the diagonalization, $C=PDP^{-1}$, we know that we can choose $P$ to be an orthogonal matrix.

It belongs to a larger class of matrix known as Hermitian matrix that guarantees that they can be diagonalized.

answered Feb 16, 2018 at 8:57

Siong Thye Goh

6,6213 gold badges19 silver badges29 bronze badges

$\begingroup$ what if there are repeated eigenvalues? how do you ensure the orthogonality then? $\endgroup$
– figs_and_nuts
18 hours ago
1

$\begingroup$ It doesn't require the eigenvalues to be distinct. One possible way to prove it is via Schur's theorem. I was introduced to the theorem via mathematical induction many years ago. $\endgroup$
– Siong Thye Goh
17 hours ago

Add a comment |

1 Answer 1