10
$\begingroup$

Intrigued by a question at math.stackexchange, and investigating it empirically, I am wondering about the following statement on the square-root of sums of i.i.d. random variables.

Suppose $X_1, X_2, \ldots, X_n$ are i.i.d. random variables with finite non-zero mean $\mu$ and variance $\sigma^2$, and $\displaystyle Y=\sum_{i=1}^n X_i$. The central limit theorem says $\displaystyle \dfrac{Y - n\mu}{\sqrt{n\sigma^2}} \ \xrightarrow{d}\ N(0,1)$ as $n$ increases.

If $Z=\sqrt{|Y|}$, can I also say something like $\displaystyle \dfrac{Z - \sqrt{n |\mu|-\tfrac{\sigma^2}{4|\mu|}}}{\sqrt{\tfrac{\sigma^2}{4|\mu|}}}\ \xrightarrow{d}\ N(0,1)$ as $n$ increases?

For example, suppose the $X_i$ are Bernoulli with mean $p$ and variance $p(1-p)$, then $Y$ is binomial and I can simulate this in R, say with $p=\frac13$:

set.seed(1)
cases <- 100000
n <- 1000
p <- 1/3
Y <- rbinom(cases, size=n, prob=p)
Z <- sqrt(abs(Y))

which gives approximately the hoped-for mean and variance for $Z$

> c(mean(Z), sqrt(n*p - (1-p)/4))
[1] 18.25229 18.25285
> c(var(Z), (1-p)/4)
[1] 0.1680012 0.1666667

and a Q-Q plot which looks close to Gaussian

qqnorm(Z)

enter image description here

$\endgroup$
  • 1
    $\begingroup$ @MichaelM: Thanks for those comments. I had started with the $X_i$ non-negative, but I thought the intuitive asymptotic behaviour you describe allowed a generalisation to more distributions. My surprises were (a) the variance of the square root of the sum apparently tending to a constant not depending on $n$ and (b) the appearance of a distribution which looks very close to Gaussian. A counter-example would be welcome, but when I tried other cases which initially seemed non-Gaussian, increasing $n$ further seemed to bring the distribution back to a CLT-type result. $\endgroup$ – Henry Oct 21 '16 at 12:00
  • $\begingroup$ A corollary of this is the root-mean-square (or quadratic mean) of i.i.d. random variables suitably scaled (multiply by $\sqrt{n}$ as with an arithmetic mean) also converges to a Gaussian distribution provided that the $4$th moment of the underlying distribution is finite. $\endgroup$ – Henry Oct 26 '16 at 20:13
  • 3
    $\begingroup$ Just a short comment: the claim is a special case of the Delta method, see Theorem 5.5.24 in the book "Statistical inference" by Casella & Berger. $\endgroup$ – Michael M Oct 30 '16 at 10:34
  • $\begingroup$ @Michael: Perhaps you see something that I am not at the moment, but I do not think this particular problem fits within the assumptions of the classical Delta method (e.g., as stated in the theorem you reference). Note that $Y$ does not converge in distribution (nontrivially on $\mathbb R$) and so "applying the Delta method with $g(y) = \sqrt{|y|}$" does not satisfy the requisite requirements. However, as S. Catterall's answer demonstrates, it provides a useful heuristic which leads to the correct answer. $\endgroup$ – cardinal Jul 22 '17 at 21:13
  • $\begingroup$ (I believe you could adapt the proof of the Delta method to cases similar to the above in order to make fully rigorous the aforementioned heuristic.) $\endgroup$ – cardinal Jul 22 '17 at 21:15
13
+100
$\begingroup$

The convergence to a Gaussian is indeed a general phenomenon.

Suppose that $X_1,X_2,X_3,...$ are IID random variables with mean $\mu\gt 0$ and variance $\sigma^2$, and define the sums $Y_n=\sum_{i=1}^n X_i$. Fix a number $\alpha$. The usual Central Limit Theorem tells us that $P(\frac{Y_n-n\mu}{\sigma\sqrt n}\leq \alpha)\to\Phi(\alpha)$ as $n\to\infty$, where $\Phi$ is the standard normal cdf. However, the continuity of the limiting cdf implies that we also have $$P\Big(\frac{Y_n-n\mu}{\sigma\sqrt n}\leq \alpha+\frac{\alpha^2 \sigma^2}{4\mu\sigma\sqrt n}\Big)\to\Phi(\alpha)$$ because the additional term on the right hand side of the inequality tends to zero. Rearranging this expression leads to $$P\Big(Y_n\leq (\frac{\alpha\sigma}{2\sqrt \mu}+\sqrt{n\mu})^2\Big)\to\Phi(\alpha)$$

Taking square roots, and noting that $\mu\gt 0$ implies that $P(Y_n\lt 0)\to 0$, we obtain $$P\Big(\sqrt{|Y_n|}\leq \frac{\alpha\sigma}{2\sqrt \mu}+\sqrt{n\mu}\Big)\to\Phi(\alpha)$$ In other words, $\frac{\sqrt{|Y_n|}-\sqrt{n\mu}}{\sigma/{2\sqrt\mu}}\xrightarrow{d}N(0,1)$. This result demonstrates convergence to a Gaussian in the limit as $n\to\infty$.

Does this mean that $\sqrt{n\mu}$ is a good approximation to $E[\sqrt{|Y_n|}]$ for large $n$? Well, we can do better than this. As @Henry notes, assuming everything is positive, we can use $E[\sqrt{Y_n}]=\sqrt{E[Y_n]-\text{Var}(\sqrt{Y_n})}$, together with $E[Y_n]=n\mu$ and the approximation $\text{Var}(\sqrt{Y_n})\approx \frac{\sigma^2}{4\mu}$, to obtain the improved approximation $E[\sqrt{|Y_n|}]\approx\sqrt{n\mu- \dfrac{\sigma^2}{4\mu}}$ as stated in the question above. Note also that we still have $$\frac{\sqrt{|Y_n|}-\sqrt{n\mu-\frac{\sigma^2}{4\mu}}}{\sigma/{2\sqrt\mu}}\xrightarrow{d}N(0,1)$$ because $\sqrt{n\mu-\frac{\sigma^2}{4\mu}}-\sqrt{n\mu}\to 0$ as $n\to\infty$.

$\endgroup$
  • $\begingroup$ You may need to add $\sqrt{n \mu}-\sqrt{n \mu-\tfrac{\sigma^2}{4\mu}} \to 0$ as ${n \to \infty}$ to get my result $\endgroup$ – Henry Oct 24 '16 at 21:44
  • $\begingroup$ @Henry You can replace $\sqrt{n\mu}$ with $\sqrt{n\mu-k}$ for any constant $k$ and this won't change the limiting distribution, but it may change the degree to which $\frac{\sqrt{|Y_n|}-\sqrt{n\mu-k}}{\sigma/{2\sqrt\mu}}$ is a good approximation to $N(0,1)$ for specific large $n$ . How did you come up with $\sqrt{n \mu-\tfrac{\sigma^2}{4\mu}}$? $\endgroup$ – S. Catterall Oct 25 '16 at 9:44
  • $\begingroup$ We have $\text{Var}(Z)=E[Z^2]-(E[Z])^2$ so $E[Z]=\sqrt{E[Z^2]-\text{Var}(Z)}$. Assuming everything is positive, $E[Z^2]=E[Y]=n\mu$ while the denominator of $\frac{\sqrt{|Y_n|}-\sqrt{n\mu}}{\sigma/{2\sqrt\mu}}$ suggests $\text{Var}(Z) \approx \dfrac{\sigma^2}{4\mu}$, and combining these leads to $E[Z] \approx \sqrt{n\mu- \dfrac{\sigma^2}{4\mu}}$. $\endgroup$ – Henry Oct 25 '16 at 14:51
  • $\begingroup$ Ok, thanks, I've tried to cover this in my answer now. $\endgroup$ – S. Catterall Oct 25 '16 at 21:25
  • $\begingroup$ (+1) Very nice answer. Excellent, rigorous demonstration. $\endgroup$ – cardinal Jul 22 '17 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.