# Equal variance along left and right singular vectors?

Please confirm or reject my line of reasoning:

Given SVD of $$X$$: $$X_{NxP}=U_{NxP}D_{PxP}V_{PxP}'$$,

Variance along ith column vector of $$U$$ is given by $$||X'u_i||^2=u'_iXX'u_i=u'_id_i^2u_i=d_i^2$$, where $$d_i$$ is ith singular value and second equality comes from eigendecomposition of $$XX'=UD^2U'$$. I omit division by $$N$$ as it doesn't affect the result.

Variance along ith column vector of $$V$$ is given by $$||Xv_i||^2=v'_iX'Xv_i=v'_id_i^2u_i=d_i^2$$

But in such a case the variance along principal components (columns of $$V$$) and variance along left singular vectors (columns of $$U$$) is equal up to a factor (N or P). Is this true?

If this is not the case, how can I argue that lower singular value implies lower variance along corresponding left singular vector (relative to other left singular vectors)? In particular, I want to confirm the following passage from Elements of Statistical Learning, p.67:

small singular values dj correspond to directions in the column space of X having small variance, and ridge regression shrinks these directions the most.

Where ridge regression with respect to SVD is given by: $$X\beta^{Ridge}=X(X'X +\lambda I)^{-1}X'y=UD(D^2+'\lambda I)^{-1}DU'y=\sum_{j=1}^{P}u_j\frac{d_j^2}{d_j^2+\lambda}u_j'y$$

It is clear that this is the case for principal components $$v_j$$ (proven in text) but the above expression shrinks left singular vectors $$u_j$$ thus the question.

• Indeed, that is true. The "variance" (in your sense, not in the sense of Var(X) where X is an r.v.) is encoded in the singular values. The U and V matrices have orthonormal columuns, in particular, they all have unit $\|\cdot\|_2$-norm. – Michael Oct 6 at 11:16