What is the duality relationship between eigensystems of X X' vs X' X? I want to know the relationship between the eigensystems of two non-negative-definite (covariance-like) matrices.  Both are derived from X which is a T-by-K real matrix (wlog say K > T).  I avoid subtracting the mean to avoid complicating the main issue which is about the eigensystem structure.
Ct = (1/T) * X' X = Et Lt Et'
Ck = (1/K) * X X' = Ek Lk Ek'
.... where Et, Lt are the (column) eigenvectors and diagonal matrix of eigenvalues respectively for Ct, and Ek Lk are the same for Ck.
I remember vaguely from school that there is some kind of duality relationship here (between the two eigensystems) which is particularly interesting because both matrices have rank T < K, and hence only T of the diagonal elements of Lt are nonzero.
Does anyone know this relationship explicitly? Thanks in advance for any light you can shed!
 A: This is most readily analyzed by means of the singular value decomposition
$$X = U \Sigma V^\prime$$
where the columns of $U$ are mutually orthonormal, so are the columns of $V$, and $\Sigma$ has nonzero values only on its diagonal $\Sigma_{ii}$. This immediately yields the decompositions
$$X^\prime X = V\Sigma^\prime\Sigma V^\prime$$
and 
$$X X^\prime = U \Sigma\Sigma^\prime U^\prime.$$
Since $V^{-1} = V^\prime$ and $U^{-1}=U^\prime$ and both $\Sigma^\prime\Sigma$ and $\Sigma\Sigma^\prime$ are square diagonal matrices, both of these equations are eigendecompositions.  (That demonstrates $\Sigma$ is uniquely determined by $X$ up to a discrete set of permutations.)  The diagonal elements of these latter two matrices--which are the eigenvalues--are the set of $\Sigma_{ii}^2$, with the same multiplicities.  When $X$ is not square, the larger of  $\Sigma^\prime\Sigma$ and $\Sigma\Sigma^\prime$ will have additional zeros to complete its diagonal.
There can be no universal relationship between $U$ (whose columns are eigenvectors of $XX^\prime$) and $V$ (whose columns are eigenvectors of $X^\prime X$), because we could start with arbitrary matrices $U$, $\Sigma$, and $V$ satisfying the SVD requirements and construct an example $X = U\Sigma V^\prime$ having these particular, but arbitrary, $U$ and $V$.
