Equation 3.49 from Elements of Statistical Learning I was on page 66 of ESL. I don't know how equation 3.49 on that page is derived. Where does the $N$ in the denominator come from ? Can someone kindly show me the steps in between the lines of equation 3.49? In particular, why is $Var(Xv_1) = \frac{d^2_1}{N}$ ? And where does the $N$ come from ? Thanks.  

 A: In the sentence before (3.48), the authors are effectively defining the term "sample covariance" to mean $\mathrm{Var}(\mathbf Y) \equiv \mathbf Y^T\mathbf Y/N$ when $\mathbf Y$ is an $N\times q$ matrix of $N$ samples in $\mathbb R^q$.
The sample covariance of $\mathbf X$ is by definition $\mathbf S=\mathrm{Var}(\mathbf X) = \mathbf X^T\mathbf X/N$, which is a $p\times p$ matrix. The $(i,j)$-th entry in this matrix is (a certain scaling of) the covariance between the $i$th and $j$th components of the $N$ samples.
The sample covariance of $\mathbf z_1 = \mathbf X v_1$ is, again by definition, $s_1=\mathrm{Var}(\mathbf z_1)=\mathbf z_1^T\mathbf z_1/N$. This is just a $1\times1$ matrix, or a scalar. We compute
$$Ns_1=(\mathbf X v_1)^T\mathbf X v_1=v_1^T\mathbf X^T\mathbf X v_1=v_1^T\mathbf V\mathbf D^2\mathbf V^Tv_1$$
$$=(1,0,\ldots,0)\mathbf D^2(1,0,\ldots,0)^T=\mathbf D^2_{1,1}=d_1^2$$
and so
$$s_1=\frac{d_1^2}{N}.$$
As for why $N$ is introduced in the denominator, consider what would happen if we sampled $N$ more data points, which happen to be the same as the first $N$. Then we're stacking two copies of $\mathbf X$ on top of each other. To recompute the SVD, start with the same $\mathbf D$ and stack two copies of $\mathbf U$ on top of each other. But this isn't really an SVD, because the columns of $\mathbf U$ are no longer orthonormal. We want $\mathbf U^T\mathbf U=\mathbf I$, but we get $2\mathbf I$. To restore orthonormality, we have to divide $\mathbf U$ by $\sqrt 2$, and to compensate, we have to multiply $\mathbf D$ and in particular $d_1$ by $\sqrt 2$. So $d_1^2$ has increased by a factor of $2$, just because we've sampled more data points. This is an undesirable artifact of the sample size, which is eliminated by considering $d_1^2/N$ instead.
That was a little hand-wavy, but I think you get the idea. :)
