7
$\begingroup$

Let $X_1,X_2,\cdots ,X_j \cdots $ be i.i.d. $\mathcal N(0, 1)$ random variables. Show that for any $a > 0$, $$\lim_{n\to \infty}P\left( \sum_{i=1}^n X_i^2\leq a\right)=0$$ It is clear that $\sum_{i=1}^n X_i^2 \sim \chi ^2_n$; then, I can't proceed. Please help.

$\endgroup$
  • $\begingroup$ write out the distribution function. It will depend on $a$ and $n$ and the gamma function. $\endgroup$ – probabilityislogic Aug 5 '12 at 5:12
  • 5
    $\begingroup$ So much homework, so little time.... $\endgroup$ – Dilip Sarwate Aug 5 '12 at 11:43
  • 4
    $\begingroup$ Hint: This has absolutely nothing to do with the normal distribution. In fact, this holds for any distribution that is not a point-mass at zero. $\endgroup$ – cardinal Aug 5 '12 at 13:08
3
$\begingroup$

Define $Y_n = \sum_{i=1} X_n^2$. As you mentioned, $Y_n$ is distributed according to chi-squared with density:

$$p(y) = C(n)x^{n/2-1}e^{-x/2}$$

where $C(n)$ is some normalization constant (defined only for $y \ge 0$).

The probability of $Y_n$ being larger than $a$ is (as a function of $n$):

$$\int_0^a C(n) x^{n/2-1} e^{-x/2}$$

where $C(n) = 2^{n/2} \Gamma(n/2)$.

This is strictly smaller than $$\int_0^a C(n) x^{n/2-1} = C(n) \frac{2}{n} a^{n/2}$$ because $e^{-z} < 1$ for $z > 0$.

This means that we need to show that

$$D(n) = \frac{2}{n 2^{n/2} \Gamma(n/2)} a^{n/2} \le \frac{d^n}{\Gamma(n/2)}$$

goes to 0 as $n$ goes to infinity and $d = \sqrt{a/2}$.

Then, $\Gamma(n/2)$ is always larger than $\Gamma(m/2)$ where $m$ is either $n$ if $n$ is even or $n-1$ if $n$ is odd (i.e. $m$ is always an even number smaller than $n$).

Therefore, $\Gamma(n/2) \le \Gamma(m/2) = (m/2-1)!$.

So we get

$$D(n) \le \frac{d^n}{(m/2-1)!}$$

which clearly goes to 0 at the speed of light.

$\endgroup$
13
$\begingroup$

The stated result has nothing to do with the normal distribution and is fully general. It can also be proven using only the most basic properties of cumulative distribution functions. In particular, it is unnecessary to appeal to (relatively) "high-powered" theorems like the Law of Large Numbers.

Proposition: Let $X_i \sim F$ be iid with any distribution other than $\delta_0$, a point-mass at zero. Then, for each $a \geq 0$, $$ \lim_{n\to\infty} \mathbb P\left(\sum_{i=1}^n X_i^2 \leq a\right) = 0 \>. $$

Define $Y_i = X_i^2$ and denote the distribution of $Y_i$ by $G$. If $F$ is of unbounded support, then so is $G$. In this case, everything is completely straightforward once we have the following lemma.

Lemma: If $Y_i \geq 0$ are iid, then $\mathbb P(\sum_{i=1}^n Y_i \leq a) \leq G^n(a)$.
Proof: $\{Y_1 + Y_2 + \dots + Y_n \leq a\} \subset \{Y_1 \leq a, Y_2 \leq a, \ldots, Y_n \leq a\}$. Hence, $$ \mathbb P\left( \sum_{i=1}^n Y_i \leq a\right) \leq G^n(a). $$ In words, if the sum of $n$ nonnegative terms is less than $a$, then all of the terms must be less than $a$. The iid property of the $Y_i$ variables is then invoked to obtain the result.

Now, if $G$ is of unbounded support, then $G(a) < 1$ for all $a \geq 0$, but then $G^n(a) \to 0$, so invoking the lemma, we are done.

Extending to the case of bounded support is not much more difficult. Suppose there exists $B > 0$ such that $G(a) = 1$ for all $a \geq B$ and $G(a) < 1$ for $a < B$. The case where $a < B$ is already handled by the argument above. For $a > B$, there is a fixed $N := N(a) = [1+(a/B)]$ such that $$ G_N(a) := \mathbb P\left(\sum_{i=1}^N Y_i \leq a\right) < 1 \>. $$ (Why?)

But, then this reduces to the previous case since
$(G_N(a))^m \to 0$ as $m \to \infty$ by considering sums over blocks of size $N$.

NB The intuition in the bounded case is that once we add enough terms, the support of the distribution of the sum will eventually catch up to, and overtake, $a$. Once that happens, we find ourselves in the previous case.

Epilogue The specific case of the normal distribution falls under the category of $F$ (hence, $G$) with unbounded support. So, we only need the first part of the answer (which requires no calculation whatsoever) to establish the result in the question statement.

$\endgroup$
  • $\begingroup$ This is a correct and detailed answer. The key idea was given in Doug Zare's answer which he mysteriously deleted. $\endgroup$ – Michael Chernick Aug 6 '12 at 10:44
  • $\begingroup$ @Michael: Thanks. That's strange. I never saw Douglas' answer; maybe with your 10K rep you can see these. $\endgroup$ – cardinal Aug 6 '12 at 11:27
  • 1
    $\begingroup$ It might be a 10K privilege. I can see the deleted answer. It went like this: "You don't need the exact distribution. For the sum of the squares to be at most $a$, each random variable has to be between $-\sqrt{a}$ and $\sqrt{a}$. Each satisfies this with some probability $p(a) \lt 1$. The chance that all $n$ are in that interval is $p(a)^n$ by independence, which goes to $0$ when $n \to \infty$, and this is an upper bound for the chance that the sum is at most $a$." Of course he needs that assumption of unbounded support or support exceeding out beyond a for the argument to work. $\endgroup$ – Michael Chernick Aug 6 '12 at 11:38
  • $\begingroup$ @Michael: Yes, I wonder why he deleted that. It's nice and compact and the argument follows through for any distribution with unbounded support. You need my final little tweak in case the support of the distribution is bounded. $\endgroup$ – cardinal Aug 6 '12 at 11:46
2
$\begingroup$

We note that since each $X_i$ has mean 0 and variance 1, $(\sum_i X_i^2)/n$ converges to 1 a.s.

But if $$ \lim_{n\to\infty} \mathbb P\Big( \sum_i X_i^2 < a \Big) > 0 \,, $$ then with positive probability $$ \frac{1}{n}\sum_i X_i^2 $$ goes to 0, which is a contradiction because if the sum have a positive probability of staying finite then dividing by n we have $(\sum_i X_i^2)/n$ converging to 0 with positive probability contradicting convergence to 1 almost surely.

$\endgroup$
  • $\begingroup$ not sure why the latter statement is true. It could be that that $\sum X_i^2$ has a small probability of being a very large number, but that number is so large (say $2^n$ with probability $1/n$) such that $\sum_{i} X_i^2 / n$ still diverges. $\endgroup$ – singelton Aug 5 '12 at 13:08
  • 1
    $\begingroup$ Almost sure convergence is based on a theorem clled the strong law of large numbers. But if ∑X$_i$$^2$ has positive probability to have a finite sum then (∑X$_i$$^2$)/n converges to 0 with positive probability contradicting the strong law. $\endgroup$ – Michael Chernick Aug 5 '12 at 13:41
  • 1
    $\begingroup$ (+1) This answer should also make it clear why the hint I provided is true. By the way, you had some unconventional, but remarkably clever, ways of typesetting the math. I've made it a bit more conventional. You might look at the edits to see what was changed. $\endgroup$ – cardinal Aug 5 '12 at 13:53
  • $\begingroup$ @singelton: The only potential subtlety here (assuming you have the SLLN in your arsenal to begin with), is whether or not the limit $\lim_n \mathbb P(\sum_{i=1}^n X_i^2 < a)$ exists in the first place. But, this follows by continuity from above since the sets $A_n := \{\omega: \sum_i X_i^2(\omega) < a \}$ are decreasing. $\endgroup$ – cardinal Aug 5 '12 at 19:09
  • $\begingroup$ I got it. I missed the "a.s. convergence" part, which means it can't converge to 0 with non-zero probability. $\endgroup$ – singelton Aug 5 '12 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.