2
$\begingroup$

This is Ex.2.4 from The Elements of Statistical Learning. enter image description here

I don’t understand the sentence that I underline in the image.

I know that $\sqrt{10}$ is approximately equal to 3.1, but I don’t know how to get it.

I write my thoughts in the below image: enter image description here

This may be a silly problem. I need your help😭

Thank you!

$\endgroup$
1
  • $\begingroup$ The expected squared distance is the sum of the expected squared components of the vector, each of which has expectation $1$ (the variance stipulated by declaring the variance matrix is "$I_p$"). $\endgroup$
    – whuber
    Commented Sep 18, 2023 at 19:32

1 Answer 1

1
$\begingroup$

On one hand, with the notations given in the exercise, you are not supposed to find $\operatorname{Var}(\|x_0\|)$, but $E[\|x_0\|^2]$ given $x_0 \sim N(0, I_p)$ (read the second paragraph of the exercise more carefully). From what you attempted I can see you have no difficulty to show that $E[\|x_0\|^2] = p$.

On the other hand, I agree with you that the underlined sentence is ambiguous in that the distance from a randomly drawn point to origin is $\|x_0\|$. Therefore, the underlined sentence seems to suggest $E[\|x_0\|] = \sqrt{p}$, which is not true. In fact, since $\|x_0\| \sim \chi_p$ (i.e., the $\chi$ (not squared!) distribution), its expectation is: \begin{align} E[\|x_0\|] = \sqrt{2}\frac{\Gamma\left(\frac{p + 1}{2}\right)}{\Gamma\left(\frac{p}{2}\right)}. \end{align} When $p = 10$, $E[\|x_0\|] = \sqrt{2}\frac{\Gamma(5.5)}{\Gamma(5)} \approx 3.08433$, which is strictly less than $\sqrt{10} \approx 3.1623$. Also note that the quoted standard deviation corresponds to the scale parameter $\sigma$ in the distributional form of $X$. In the original problem statement, $\sigma$ was assumed to be $1$. The verbal interpretation in the third paragraph would be easier to understand if at the outset the distribution of $X$ is assumed to be $N(0, \sigma^2I_p)$. Under this condition, $E[\|x_0\|^2] = p\sigma^2$ and $E[\|x_0\|] = \sqrt{2}\frac{\Gamma\left(\frac{p + 1}{2}\right)}{\Gamma\left(\frac{p}{2}\right)}\sigma$ still hold.

Therefore, I think the author failed to connect the mathematical result in the second paragraph and the verbal interpretation in the third paragraph in a precise manner. In my opinion, the underlined sentence needs to be modified to either a randomly drawn test point is about 3.08 standard deviations from the origin or the square root of the average squared distance of a randomly drawn test point from the origin is about 3.16 standard deviations.

$\endgroup$
1
  • $\begingroup$ Thanks for your answer! I have understood now! I had a misunderstanding of the question before and have been struggling with calculating the standard deviation of ||x_0||😂 $\endgroup$
    – chenqile
    Commented Sep 19, 2023 at 5:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.