# The test point’s standard deviation from the origin (ESL’s Exercise)

This is Ex.2.4 from The Elements of Statistical Learning.

I don’t understand the sentence that I underline in the image.

I know that $$\sqrt{10}$$ is approximately equal to 3.1, but I don’t know how to get it.

I write my thoughts in the below image:

This may be a silly problem. I need your help😭

Thank you!

• The expected squared distance is the sum of the expected squared components of the vector, each of which has expectation $1$ (the variance stipulated by declaring the variance matrix is "$I_p$").
– whuber
Commented Sep 18, 2023 at 19:32

## 1 Answer

On one hand, with the notations given in the exercise, you are not supposed to find $$\operatorname{Var}(\|x_0\|)$$, but $$E[\|x_0\|^2]$$ given $$x_0 \sim N(0, I_p)$$ (read the second paragraph of the exercise more carefully). From what you attempted I can see you have no difficulty to show that $$E[\|x_0\|^2] = p$$.

On the other hand, I agree with you that the underlined sentence is ambiguous in that the distance from a randomly drawn point to origin is $$\|x_0\|$$. Therefore, the underlined sentence seems to suggest $$E[\|x_0\|] = \sqrt{p}$$, which is not true. In fact, since $$\|x_0\| \sim \chi_p$$ (i.e., the $$\chi$$ (not squared!) distribution), its expectation is: \begin{align} E[\|x_0\|] = \sqrt{2}\frac{\Gamma\left(\frac{p + 1}{2}\right)}{\Gamma\left(\frac{p}{2}\right)}. \end{align} When $$p = 10$$, $$E[\|x_0\|] = \sqrt{2}\frac{\Gamma(5.5)}{\Gamma(5)} \approx 3.08433$$, which is strictly less than $$\sqrt{10} \approx 3.1623$$. Also note that the quoted standard deviation corresponds to the scale parameter $$\sigma$$ in the distributional form of $$X$$. In the original problem statement, $$\sigma$$ was assumed to be $$1$$. The verbal interpretation in the third paragraph would be easier to understand if at the outset the distribution of $$X$$ is assumed to be $$N(0, \sigma^2I_p)$$. Under this condition, $$E[\|x_0\|^2] = p\sigma^2$$ and $$E[\|x_0\|] = \sqrt{2}\frac{\Gamma\left(\frac{p + 1}{2}\right)}{\Gamma\left(\frac{p}{2}\right)}\sigma$$ still hold.

Therefore, I think the author failed to connect the mathematical result in the second paragraph and the verbal interpretation in the third paragraph in a precise manner. In my opinion, the underlined sentence needs to be modified to either a randomly drawn test point is about 3.08 standard deviations from the origin or the square root of the average squared distance of a randomly drawn test point from the origin is about 3.16 standard deviations.

• Thanks for your answer! I have understood now! I had a misunderstanding of the question before and have been struggling with calculating the standard deviation of ||x_0||😂 Commented Sep 19, 2023 at 5:57