11
$\begingroup$

Let $X_1,\dots,X_n \sim U(0,1)$ be independent and identicallly distributed standard uniform random variables.

$$\text{Let }\quad Y_n=\sum_i^nX_i^2 \quad \quad \text{I seek: } \quad \mathbb{E}\big[\sqrt{Y_n } \big]$$

The expectation of $Y_n$ is easy:

$$\begin{align} \mathbb{E}\left[X^2\right] &=\int_0^1\frac{y}{2\sqrt{y}}=\frac{1}{3}\\ \mathbb{E}\left[Y_n\right] &=\mathbb{E}\left[\sum_i^nX_i^2\right] = \sum_i^n\mathbb{E}\left[X_i^2\right]=\frac{n}{3} \end{align}$$

Now for the boring part. To apply LOTUS, I would need the pdf of $Y_n$. Of course the pdf of the sum of two independent random variables is the convolution of their pdfs. However, here we have $n$ random variables and I guess the convolution would lead to a...convoluted expression (horrible pun intended). Is there a smarter way?

I would prefer to see the correct solution, but if it's impossible or too complicated, an asymptotic approximation for large $n$ could be acceptable. By Jensen's inequality, I know that

$$\sqrt{\mathbb{E}[Y_n]}=\sqrt{\frac{n}{3}}\geq\mathbb{E}\left[\sqrt{Y_n}\right]$$

But this doesn't help me much, unless I can find also a non-trivial lower bound. Note that the CLT doesn't directly apply here, because we have the square root of the sum of independent RVs, not just the sum of independent RVs. Maybe there could be other limit theorems (which I ignore) that may be of help here.

Improve this question
$\endgroup$
21
  • 3
    $\begingroup$ See this question for asymptotic result: stats.stackexchange.com/questions/241504/… $\endgroup$
    – S. Catterall
    Dec 4, 2017 at 18:56
  • 4
    $\begingroup$ I get $\mathbb{E}[\sqrt{Y_n}]\approx \sqrt{\frac{n}{3}-\frac{1}{15}}$ based on the above linked question. $\endgroup$
    – S. Catterall
    Dec 4, 2017 at 20:10
  • 2
    $\begingroup$ I don't think I would use any of the approaches described in that answer (of which there are more than two!) :-). The reason is that you can avail yourself of simple, straightforward simulations to estimate the expectations, while an analytical solution seems unobtainable. I like @S.Catterall's approach very much (+1 for that solution, which I hadn't read before). Simulation shows it works well even for small $n$. $\endgroup$
    – whuber ♦
    Dec 4, 2017 at 21:31
  • 3
    $\begingroup$ The simulation is worth doing :-). Plot the difference between the simulated mean and the approximate formula against $n$. It will show you clearly how well the approximation works as a function of $n$. $\endgroup$
    – whuber ♦
    Dec 4, 2017 at 23:57
  • 4
    $\begingroup$ Clearly $E\left[\sqrt{Y_1}\right]=0.5$ while the approximation gives $\sqrt{\frac{1}{3}-\frac{1}{15}} = \sqrt{\frac4{15}}\approx 0.516$. In that case $\sqrt{\frac{1}{3}-\frac{1}{12}}$ would have been correct. But the approximation improves after that. $\endgroup$
    – Henry
    Dec 5, 2017 at 9:12

2 Answers 2

Reset to default
13
$\begingroup$

One approach is to first calculate the moment generating function (mgf) of $Y_n$ defined by $Y_n = U_1^2 + \dotsm + U_n^2$ where the $U_i, i=1,\dotsc, n$ is independent and identically distributed standard uniform random variables.

When we have that, we can see that $$ \DeclareMathOperator{\E}{\mathbb{E}} \E \sqrt{Y_n} $$ is the fractional moment of $Y_n$ of order $\alpha=1/2$. Then we can use results from the paper Noel Cressie and Marinus Borkent: "The Moment Generating Function has its Moments", Journal of Statistical Planning and Inference 13 (1986) 337-344, which gives fractional moments via fractional differentiation of the moment generating function.

First the moment generating function of $U_1^2$, which we write $M_1(t)$. $$ M_1(t) = \E e^{t U_1^2} = \int_0^1 \frac{e^{tx}}{2\sqrt{x}}\; dx $$ and I evaluated that (with help of Maple and Wolphram Alpha) to give $$ \DeclareMathOperator{\erf}{erf} M_1(t)= \frac{\erf(\sqrt{-t})\sqrt{\pi}}{2\sqrt{-t}} $$ where $i=\sqrt{-1}$ is the imaginary unit. (Wolphram Alpha gives a similar answer, but in terms of the Dawson integral.) It turns out we will mostly need the case for $t<0$. Now it is easy to find the mgf of $Y_n$: $$ M_n(t) = M_1(t)^n $$ Then for the results from the cited paper. For $\mu>0$ they define the $\mu$th order integral of the function $f$ as $$ I^\mu f(t) \equiv \Gamma(\mu)^{-1} \int_{-\infty}^t (t-z)^{\mu-1} f(z)\; dz $$ Then, for $\alpha>0$ and nonintegral, $n$ a positive integer, and $0<\lambda<1$ such that $\alpha=n-\lambda$. Then the derivative of $f$ of order $\alpha$ is defined as $$ D^\alpha f(t) \equiv \Gamma(\lambda)^{-1}\int_{-\infty}^t (t-z)^{\lambda-1} \frac{d^n f(z)}{d z^n}\; dz. $$ Then they state (and prove) the following result, for a positive random variable $X$: Suppose $M_X$ (mgf) is defined. Then, for $\alpha>0$, $$ D^\alpha M_X(0) = \E X^\alpha < \infty $$ Now we can try to apply these results to $Y_n$. With $\alpha=1/2$ we find $$ \E Y_n^{1/2} = D^{1/2} M_n (0) = \Gamma(1/2)^{-1}\int_{-\infty}^0 |z|^{-1/2} M_n'(z) \; dz $$ where the prime denotes the derivative. Maple gives the following solution: $$ \int_{-\infty}^0 \frac{n\cdot\left(\erf(\sqrt{-z})\sqrt{\pi}-2e^z\sqrt{-z} \right)e^{\frac{n(-2\ln 2 +2 \ln(\erf(\sqrt{-z}))-\ln(-z) +\ln(\pi))}{2}}}{2\pi(-z)^{3/2}\erf(\sqrt{-z})} \; dz $$ I will show a plot of this expectation, made in maple using numerical integration, together with the approximate solution $A(n)=\sqrt{n/3-1/15}$ from some comment (and discussed in the answer by @Henry). They are remarkably close:

Comparision exact and approximate

As a complement, a plot of the percentage error:

Relative error (percent) in above plot

Above about $n=20$ the approximation is close to exact. Below the maple code used:

int( exp(t*x)/(2*sqrt(x)), x=0..1 ) assuming t>0;
int( exp(t*x)/(2*sqrt(x)), x=0..1 ) assuming t<0;
M := t -> erf(sqrt(-t))*sqrt(Pi)/(2*sqrt(-t))
Mn := (t, n) -> exp(n*log(M(t)))
A  :=  n -> sqrt(n/3 - 1/15)
Ex :=  n ->   int( diff(Mn(z, n), z)/(sqrt(abs(z))*GAMMA(1/2) ), 
                   z=-infinity..0 , numeric=true)

plot([Ex(n), A(n)], n=1..100, color=[blue, red], legend= 
      [exact, approx], labels=[n, expectation], 
      title="expectation of sum of squared uniforms")
plot([((A(n)-Ex(n))/Ex(n))*100], n=1..100, color= 
      [blue], labels=[n, "% error"], 
      title="Percentage error of approximation")
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ very interesting. If you could add some plots, this would be an excellent answer. However, I'll note one distinct advantage of the CLT approximation here. The approximation shows clearly that $\mathbb{E}[\sqrt{Y_n}]$ grows as $\sqrt{n}$ when $n\to\infty$. The Maple solution doesn't (or at least I can't figure that out). $\endgroup$
    – DeltaIV
    Dec 7, 2017 at 10:09
5
$\begingroup$

As an extended comment: it seems clear here that $E\left[\sqrt{Y_n}\right]= E\left[\sqrt{\sum_i X_i^2}\right]$ starts with $E\left[\sqrt{Y_n}\right] =\frac12=\sqrt{\frac{n}{3}-\frac{1}{12}}$ when $n=1$ and then approaches $\sqrt{\frac{n}{3}-\frac{1}{15}}$ as $n$ increases, related to the variance of $\sqrt{Y_n}$ falling from $\frac{1}{12}$ towards $\frac{1}{15}$. My linked question which S.Catterall answered provides a justification for the $\sqrt{\frac{n}{3}-\frac{1}{15}}$ asymptotic result based on each $X_i^2$ having mean $\frac13$ and variance $\frac{4}{45}$, and for the distribution being approximately and asymptotically normal.

This question is effectively about the distributions of distances from the origin of random points in an $n$-dimensional unit hypercube $[0,1]^n$. It is similar to a question on the distribution of distances between points in such a hypercube, so I can easily adapt what I did there to show the densities for various $n$ from $1$ to $16$ using numerical convolution. For $n=16$, the suggested normal approximation shown in red is a good fit, and from $n=4$ you can see a bell curve appearing.

enter image description here

For $n=2$ and $n=3$ you get a sharp peak at the mode of $1$ with what looks like the same density in both cases. Compare this with the distribution of $\sum_i X_i$, where the bell curve appears with $n=3$ and where the variance is proportional to $n$

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ The almost constant variance leads to possibly counter-intuitive results. For example with $n=400$, $\sqrt{Y_{400}}$ (the distance from the origin of a random point in an $400$-dimensional unit hypercube) can take any value from $0$ to $20$ but $94\%$ of cases will be between $11$ and $12$ and virtually all between $10$ and $13$ $\endgroup$
    – Henry
    Dec 7, 2017 at 10:47
  • 1
    $\begingroup$ it is a bit counter-intuitive, in fact. Because of the curse of dimensionality, I was expecting the vast majority of the points to be close to the corners (realizations $y_{400}$ s.t. $\sqrt{y_{400}}=20$). Instead it looks like the vast majority of the points are far away from the origin, but not as far as the corners.Probably the error is that we should consider distance from the center of the hypercube, not distance from the origin, which is just one of the corners of the hypercube. $\endgroup$
    – DeltaIV
    Dec 7, 2017 at 11:28
  • 3
    $\begingroup$ @DeltaIV: If you make your hypercube side $2$ so $[-1,1]^n$ and measuring from the origin, you get precisely the same distribution, expectation and variance. With $n=400$ most points in this larger hypecube will be close to the boundary of this hypercube (typical distance of the order of $0.02$) but not close to its corners (typical distance to the nearest one $11$ or $12$ again) $\endgroup$
    – Henry
    Dec 7, 2017 at 12:19
  • 1
    $\begingroup$ that makes sense - I didn't have time to do the math, but intuitively I expected similar results for $U([-1, 1])$. I was expecting the expectation (sorry for the pun) to change by a constant factor, but as I said I didn't have time to check it. $\endgroup$
    – DeltaIV
    Dec 7, 2017 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.