12
$\begingroup$

My professor claimed that $\lim_{p\to\infty}\chi^2_p$ has a normal distribution. The claim was made on the basis of the Central Limit Theorem: as $p\to\infty$, we have a Normal$(p\mu, p^2\sigma^2)$. I do not see how this is valid nor true, as this claim would have a limit of $p$ on the left hand side, yet $p$ also appears on the right hand side. Furthermore, $\sigma^2$ and $\mu$ both depend on $p$...

What am I missing and how do I convince myself of this limit's distribution?

$\endgroup$
2
  • $\begingroup$ $\lim_{p \rightarrow \infty} \frac{\chi^2_p - p\mu}{p\sigma} \overset{d}{\rightarrow} N(0,1)$ en.wikipedia.org/wiki/… $\endgroup$
    – quester
    Oct 3, 2019 at 6:57
  • 3
    $\begingroup$ You are not missing anything, and your professor's claim is false for exactly the same reasons as you came up with: limiting operations need a fixed target and not a moving one where $p$ appears in the limit. What is true is that the distribution of a suitably unitized (zero mean, unit variance) random variable related to $\chi_p^2$ is converging to the standard normal random distribution. $\endgroup$ Oct 3, 2019 at 20:07

2 Answers 2

15
$\begingroup$

This property follows from the central limit theorem, using the fact that the chi-squared distribution is obtained as the distribution of a sum of squares of independent standard normal random variables. If you have a sequence of random variables $Z_1,Z_2,Z_3, ... \sim \text{IID N}(0,1)$ then you have:$^\dagger$

$$\chi_p^2 \equiv \sum_{i=1}^p Z_i^2 \sim \text{ChiSq}(p).$$

Now, the random variables $Z_1^2,Z_2^2,Z_3^2, ... $ are IID with mean $\mathbb{E}(Z_i^2) = 1$ and variance $\mathbb{V}(Z_i^2) = 2 < \infty$, so we have $\mathbb{E}(\chi_p^2) = p$ and $\mathbb{V}(\chi_p^2) = 2p$. Applying the classical central limit theorem you get:

$$\lim_{p \rightarrow \infty} \mathbb{P} \Bigg( \frac{\chi_p^2 - p}{\sqrt{2p}} \leqslant z \Bigg) = \Phi(z).$$

Another way of writing this formal limiting result is that:

$$\frac{\chi_p^2 - p}{\sqrt{2p}} \overset{\text{Dist}}{\rightarrow} \text{N}(0, 1).$$

That is the formal convergence result that holds for the chi-squared distribution. Informally, for large $p \in \mathbb{N}$ we have the approximate distribution:

$$\chi_p^2 \overset{\text{Approx}}{\rightarrow} \text{N}(p, 2p).$$

Though not strictly correct, sometimes this informal approximation is asserted as a kind of convergence result, informally referring to convergence where $p$ appears on both sides. (Or sometimes it is made strictly correct by adding an appropriate order term.) This is presumably what your professor was referring to.

In regard to this property, it is worth noting that the gamma distribution converges to the normal as the scale parameter tends to infinity; the convergence of the chi-squared distribution to the normal is a special case of this broader convergence result.


$^\dagger$ As a sidenote, one often proceeds more generally by writing:

$$\chi_{p}^{2} = \sum_{i = 1}^{p} \bigg( \frac{X_i - \mu_{i}}{\sigma_{i}} \bigg)^2,$$

where the $X_{i} \sim \text{N}(\mu_i, \sigma_i)$ are independent Gaussian random variables with arbitrary mean and variance. By setting $Z_i \equiv (X_i - \mu_{i})/\sigma_{i}$, we can write the above formula using standardised versions of any independent normal random variables instead.

$\endgroup$
3
  • $\begingroup$ Thanks for the answer and upvoted! Just wondering whether we could say something similar for the limit of the $\chi(p)$ distribution as $p \to \infty?$ Here $\chi(p)$ distribution is the square root of $\chi^2(p)$ distribution. $\endgroup$
    – Mathmath
    May 28, 2020 at 20:11
  • 1
    $\begingroup$ A standardised chi random variable also converges in distribution to the normal, so the limit result you get is similar, but with the mean and variance of the chi distribution replacing $p$ and $2p$ respectively. $\endgroup$
    – Ben
    May 28, 2020 at 22:11
  • $\begingroup$ Thanks but how do we prove it? We can't only use central limit theorem like in the proof of the asymptotic normality of normalized $\chi^2$ distribution, since at some point we'll need to take the square root. Where will we take this? On the other hand, the wiki page seems to point to a theorem by Fisher that states approximately, $\sqrt{2 \chi^2_m} \sim \mathcal{N}(\sqrt{2m-1},1),$ for which I didn't find an available reference, and it also doesn't follow from approx. $\chi^2_m \sim \mathcal{N}(m,2m).$ So if you've some reference of this approximate normality, I'd appreciate to have it. $\endgroup$
    – Mathmath
    Aug 19, 2020 at 18:22
2
$\begingroup$

I think I'm too late to the party, but I'm writing this answer only to find a way around the confusion caused by the appearance of $p$ on the left and the right sides, namely: in the terms of the chi-square distribution $\chi^2_p$ and the normal one $\mathcal{N}(p,2p),$ as pointed out in Ben's answer above. This point was raised in the OP and also commented upon by Dilip Sarwate in the comment section to the OP. Now here, one can still be interested in the difference of the two distributions$\chi^2_p$ and $\mathcal{N}(p,2p).$ Along this line, one can indeed guarantee here that:

$$ lim_{p \to \infty}\| \Phi_{\chi^2_p} - \Phi_{\mathcal{N}(p,2p)} \|_{L^{\infty}(\mathbb{R})}=0 .....(0)$$

where $\Phi_X$ denotes the CDF of the random variable $X.$ Note the uniform convergence of the difference above to zero, which is really the key point I'm making.

The proof is a one-step application of Berry-Eseén theorem, which translates to in our case as follows (check here that the hypothesis of this theorem is really true in our case!)

$$ | \Phi_{\frac{\chi^2_p - p}{\sqrt{2p}}}(x) - \Phi_{\mathcal{N}(0,1)}(x) | \le \frac{C}{\sqrt{p}} \forall x \in \mathbb{R}.....(1) $$

where $C$ is independent of $p.$ Note that this is stronger than CLT in this case, as the bound on the right hand side is independent of $x \in \mathbb{R},$ meaning the convergence of the difference between the two CDF's is uniform,which is not necessarily the case for convergence in distribution, as it's a pointwise convergence of the CDF's at the point of continuity of the limit CDF. Note that, the uniformity w.r.t. $x$ also implies:

$$ | \Phi_{\frac{\chi^2_p - p}{\sqrt{2p}}}(\frac{x-p}{\sqrt{2p}}) - \Phi_{\mathcal{N}(0,1)}(\frac{x-p}{\sqrt{2p}}) | \le \frac{C}{\sqrt{p}} \forall x \in \mathbb{R}.....(2) $$

Next, if we use the fact that $\forall a > 0,$

$$\Phi_{aX + b}(x) = \Phi_X(\frac{x-a}{b}) $$ $$\implies \Phi_{\chi^2_p}(x) = \Phi_{\frac{\chi^2_p - p}{\sqrt{2p}}}(\frac{x-p}{\sqrt{2p}}) , .....(3)$$

and similarly:

$$\Phi_{\mathcal{N}(p,2p)} = \Phi_{\mathcal{N}(0,1)}(\frac{x-p}{\sqrt{2p}}) .....(4)$$

Now the equations (1), (3) and (4) collectively implies the equation (0). This proves that even if the d.f. $p \to \infty,$ the difference between the CDF's of $\chi^2_p$ and $\mathcal{N}(p,2p)$ indeed goes to zero. N.B. to answer your question, we only used the fact the pointwise difference between the two CDF's in (1) is independent of the point $x$ and goes to zero as $p\to \infty,$ i.e. the convergence of the CDF's is uniform; we didn't really use the fact that the rate of convergence is $O(\frac{1}{\sqrt{p}}).$

I hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.