# Variance of a principal component

Suppose that we have a SVD for our data matrix centered $$X = UDV^T$$.

Then it is stated that the i-th principal component, $$Xv_i$$, has variance $$\frac{d_i^2}{N}$$. Consider these steps.

$$var(Xv_i) = var(UDV^Tv_i) \\ = var(d_i u_i) \\ = d_i^2 var(u_i)$$

In order to get that the variance is $$\frac{d_i^2}{N}$$, we would require $$var(u_i) = \frac{1}{N}$$. But how is it that this is always true? $$u_i$$ is a unit vector, so what if we have the unit vector $$[1 \ \ 0]^T$$, then the variance is 0.25, which is not equivalent to $$\frac{1}{2}$$?

• What is your source for this assertion? What is (trivially) true is that the mean square of $u_i$ is $1/N,$ so that's likely what the source means by "variance." – whuber Jul 17 at 14:56
• @whuber I can't recall the particular source, but I think I've seen it in a couple of places. However, I did just find a statement from Elements of Statistical Learning, 2nd ed., pg 66, Eq. (3.49). It seems to only apply to the first principal component, where $var(z_1) = var(Xv_1) = \frac{d_1^2}{N}$. This appears to suggest that the variance of the normalized first principal component is $var(u_1) = \frac{1}{N}$? And right, I understand that the mean square of $u_i$ would be $\frac{1}{N}$ since it is a unit vector. – user5965026 Jul 17 at 16:40
• @whuber However, I can't think of many situations where this would hold for $var(u_1)$. The ones I can think of are the situations where $u_1$ is even sized and there's an equal number of positive and negative $\frac{1}{\sqrt{N}}$ components, in which case the variance would be 1. The mean would be zero, and the sum of squared difference would just be the mean square. – user5965026 Jul 17 at 16:43
• For every $i,$ the variance of $u_i$ equals $1/N-|\bar u_i|^2$ where $\bar u_i$ is the mean of the components of $u_i.$ – whuber Jul 17 at 17:54
• @whuber Ah I see. The column space of $X$ is orthogonal to the 1 vector. The columns of $U$ form an orthonormal basis for the column space of $X$, implying that every vector in the basis is always orthogonal to the 1 vector. ESL seems to single out the first principal component as having variance $\frac{d_i^2}{N}$, but like you said, I think this applies to all the eigenvectors $u_i$ with a corresponding non-zero eigenvalue. – user5965026 Jul 17 at 21:39