While studying PCA, I saw this question but not able to solve. Given a data matrix X that is taller than it is wide, prove that every right singular vector of X with singular value s is an eigenvector of the covariance matrix, cov(X), with eigenvalue s2.
$\begingroup$
$\endgroup$
Add a comment
|
$\begingroup$
$\endgroup$
This is a straightforward application of definitions. Write out the right-SVD of $X$ as: $X=U\Sigma V^T$. The covariance matrix is then $XX^T = U(\Sigma\Sigma^T)U^T$. Notice that this is of the form $USU^T$, where $S$ is a square matrix. Moreover, since $\Sigma$ is tall (say, $m\times n$ for $m>n$), then $\Sigma$ breaks into a diagonal $n\times n$ matrix and a $(m-n)\times n$ matrix of zeros. So show that this implies $\Sigma\Sigma^T$ is diagonal. Now you can just read off the eigenvectors and eigenvalues of $XX^T$.
