Proof of Variance Formula for Central Chi-Squared Distribution

Question

I wanted to know what the proof for the variance term in a central chi-squared distribution (degree n) is. I know that the answer is 2n, but I was wondering how to derive it.

Here's my attempt so far:

Let $X_2$ denote a variable governed by the central chi-squared distribution.

$Var(X_2) = E[(X_2)^2] - (E[X_2])^2$

$= E[(X_2)^2] - (n^2)$

I was able to prove that the mean of a central chi-squared distribution is it's degree (n), by using the formula:

$E[Za*Za] = Cov(Za, Za) + E[Za]^2$

$= sa*sa + 0$

$= sa^2 = 1^2 = 1.$

$E[X_2] = SUM(E[Za^2])$, a goes from 1 to n. (using linearity of expectation)

$= SUM(1)$, 1 to n

= n

where,

$X_2 = SUM(Za^2)$, a goes from 1 to n, and Za ~ N(0, 1) (this is the chi-squared definition)

sa = standard deviation, which in this case, is = 1.

So using this knowledge, the only term in the variance definition for the chi-squared distribution that I don't know, is $E[(X2)^2]$. However, I'm at a loss as to how to compute this. Any help here would be greatly appreciated.

N.B. This is my first post here, and I don't know a lot about mathematical typing. Pardon my poor notation and lack of detail in the question, if any. Would be happy to answer any counter-questions.

Thanks!

Answer 1

The $k$-th moment $E[X^k]$ of a general Gamma random variable with (order, rate) parameters $(s,\lambda)$ is \begin{align} E[X^k] &= \int_0^\infty x^k\cdot \underbrace{\lambda\frac{(\lambda x)^{s-1}}{\Gamma(s)}e^{-\lambda x}}_{\Gamma(s,\lambda)~\text{density}} \,\mathrm dx\\ &= \lambda^{-k}\int_0^\infty \lambda\frac{(\lambda x)^{k+s-1}}{\Gamma(s)} e^{-\lambda x}\,\mathrm dx\\ &= \lambda^{-k} \frac{\Gamma(k+s)}{\Gamma(s)}\int_0^\infty \underbrace{\lambda\frac{(\lambda x)^{k+s-1}}{\Gamma(k+s)}e^{-\lambda x}}_{\Gamma(k+s,\lambda)~\text{density}} \,\mathrm dx\\ &= \lambda^{-k} \frac{(k+s-1)\cdot(k+s-2)\cdot~\cdots~\cdot s\cdot\Gamma(s)}{\Gamma(s)}\\ &= \frac{(k+s-1)\cdot(k+s-2)\cdot~\cdots~\cdot s}{\lambda^{k}} \end{align} Applying this to the case of a $\chi^2$ random variable with $n$ degrees of freedom which is a $\Gamma\left(\frac n2,\frac 12\right)$ random variable, we get that $$E[X] = \frac{\left(\frac n2\right)}{\frac 12} = n; \quad E[X^2] = \frac{\left(\frac n2+1\right)\left(\frac n2\right)}{\left(\frac 12\right)^2} = n^2+2n$$ and $$ \operatorname{var}(X) = E[X^2] - (E[X])^2 = 2n.$$

---

Alternatively, from the properties of standard normal random variables, \begin{align} E[Z^4] &= \int_{-\infty}^\infty z^4 f(z)\,\mathrm dz = 2\int_0^\infty z^4 \frac{1}{\sqrt{2\pi}}e^{-z^2/2}\,\mathrm dz\\ &= \frac{4}{\sqrt{\pi}}\int_0^\infty y^{3/2}e^{-y}\,\mathrm dy \qquad\scriptstyle{\text{on substituting $y$ for $z^2/2$}}\\ &= \frac{4}{\sqrt{\pi}}\Gamma\left(\frac 52\right) = \frac{4}{\sqrt{\pi}} \times \frac 32 \times \frac 12 \times \sqrt{\pi}\\ &= 3 \end{align} and so \begin{align} E\left[\left(\sum_{i=1}^n Z_i^2\right)^2 \right] &= E\left[\sum_{i=1}^n Z_i^4\right] + 2 \sum_{i=1}^n\sum_{j=i+1}^n E[Z_i^2]E[Z_j^2]\\ &= 3n + n(n-1)\\ &= n^2+2n \end{align} giving $\displaystyle\operatorname{var}\left(\sum_{i=1}^n Z_i^2\right) = E\left[\left(\sum_{i=1}^n Z_i^2\right)^2 \right] - \left(E\left[\sum_{i=1}^n Z_i^2\right]\right)^2 = 2n$ as before.

Answer 2

I was going to solve the integral with the chi squared density, but it seems that you want a derivation from the definition of a $\chi_n^2$ random variable as a sum of squares of $n$ independent standard normals (let's say $Z_i$). Working from this:

\begin{align} X^2 =& (\sum_{i=1}^nZ_i^2)^2\\ =& \sum_{i=1}^nZ_i^4+\sum_{i \neq j}^nZ_i^2Z_j^2 \end{align}

Therefore, using independence of $Z_i$s and linearity of expectation:

\begin{align} E(X^2)=& \sum_{i=1}^nE(Z_i^4)+\sum_{i \neq j}^nE(Z_i^2)E(Z_j^2)\\ =& \sum_{i=1}^nE(Z_i^4)+ n(n-1) \end{align}

Where we know that all the terms of the second sum are 1, and there are $n(n-1)$ terms. The only other thing that we need to know is the fourth moment (often called kurtosis) of the standard normal, which is 3. Since there are $n$ kurtosis terms, we have:

\begin{align} E(X^2)=& 3n + n(n - 1)\\ =& 2n+n^2 \end{align}

Answer 3

Thanks @Christopher Hanck for the hint!

My reference page: https://en.wikipedia.org/wiki/Moment-generating_function

In the table on the link above, the moment-generating function for the chi-squared distribution is given as: M(t) = (1 - 2t)^(-k/2)

Also, a very important section on the page linked to above, is the "Calculations of moments" section. This section gives the following formula:

E[X^n] = n'th derivative wrt. t of M(X, 0)

where M(X, 0) = (1 - 2t)^(-k/2) in our case

So basically, you take the derivative of the above function twice wrt. t, and then just plug in t = 0. And that's about it!

So after double differentiation wrt. t, you have,

E[X2^2] = k(k+2)(1 - 2t)^(-(k+4)/2)

Putting t = 0 above, we have

E[X2^2] = k(k + 2) = k^2 + 2k ... (for degree k)

From the question body above, we have:

var(X2) = E[(X2)^2] - (n^2)

= n^2 + 2n - n^2

= 2n

Q.E.D.

This does however, lead me to ask: How is the moment generating function of the chi-squared distribution derived?