2
$\begingroup$

I wanted to know what the proof for the variance term in a central chi-squared distribution (degree n) is. I know that the answer is 2n, but I was wondering how to derive it.

Here's my attempt so far:

Let $X_2$ denote a variable governed by the central chi-squared distribution.

$Var(X_2) = E[(X_2)^2] - (E[X_2])^2$

$= E[(X_2)^2] - (n^2)$

I was able to prove that the mean of a central chi-squared distribution is it's degree (n), by using the formula:

$E[Za*Za] = Cov(Za, Za) + E[Za]^2$

$= sa*sa + 0$

$= sa^2 = 1^2 = 1.$

$E[X_2] = SUM(E[Za^2])$, a goes from 1 to n. (using linearity of expectation)

$= SUM(1)$, 1 to n

= n

where,

$X_2 = SUM(Za^2)$, a goes from 1 to n, and Za ~ N(0, 1) (this is the chi-squared definition)

sa = standard deviation, which in this case, is = 1.

So using this knowledge, the only term in the variance definition for the chi-squared distribution that I don't know, is $E[(X2)^2]$. However, I'm at a loss as to how to compute this. Any help here would be greatly appreciated.

N.B. This is my first post here, and I don't know a lot about mathematical typing. Pardon my poor notation and lack of detail in the question, if any. Would be happy to answer any counter-questions.

Thanks!

$\endgroup$
  • $\begingroup$ have a look at moment-generating functions $\endgroup$ – Christoph Hanck Sep 15 '15 at 7:21
4
$\begingroup$

The $k$-th moment $E[X^k]$ of a general Gamma random variable with (order, rate) parameters $(s,\lambda)$ is \begin{align} E[X^k] &= \int_0^\infty x^k\cdot \underbrace{\lambda\frac{(\lambda x)^{s-1}}{\Gamma(s)}e^{-\lambda x}}_{\Gamma(s,\lambda)~\text{density}} \,\mathrm dx\\ &= \lambda^{-k}\int_0^\infty \lambda\frac{(\lambda x)^{k+s-1}}{\Gamma(s)} e^{-\lambda x}\,\mathrm dx\\ &= \lambda^{-k} \frac{\Gamma(k+s)}{\Gamma(s)}\int_0^\infty \underbrace{\lambda\frac{(\lambda x)^{k+s-1}}{\Gamma(k+s)}e^{-\lambda x}}_{\Gamma(k+s,\lambda)~\text{density}} \,\mathrm dx\\ &= \lambda^{-k} \frac{(k+s-1)\cdot(k+s-2)\cdot~\cdots~\cdot s\cdot\Gamma(s)}{\Gamma(s)}\\ &= \frac{(k+s-1)\cdot(k+s-2)\cdot~\cdots~\cdot s}{\lambda^{k}} \end{align} Applying this to the case of a $\chi^2$ random variable with $n$ degrees of freedom which is a $\Gamma\left(\frac n2,\frac 12\right)$ random variable, we get that $$E[X] = \frac{\left(\frac n2\right)}{\frac 12} = n; \quad E[X^2] = \frac{\left(\frac n2+1\right)\left(\frac n2\right)}{\left(\frac 12\right)^2} = n^2+2n$$ and $$ \operatorname{var}(X) = E[X^2] - (E[X])^2 = 2n.$$


Alternatively, from the properties of standard normal random variables, \begin{align} E[Z^4] &= \int_{-\infty}^\infty z^4 f(z)\,\mathrm dz = 2\int_0^\infty z^4 \frac{1}{\sqrt{2\pi}}e^{-z^2/2}\,\mathrm dz\\ &= \frac{4}{\sqrt{\pi}}\int_0^\infty y^{3/2}e^{-y}\,\mathrm dy \qquad\scriptstyle{\text{on substituting $y$ for $z^2/2$}}\\ &= \frac{4}{\sqrt{\pi}}\Gamma\left(\frac 52\right) = \frac{4}{\sqrt{\pi}} \times \frac 32 \times \frac 12 \times \sqrt{\pi}\\ &= 3 \end{align} and so \begin{align} E\left[\left(\sum_{i=1}^n Z_i^2\right)^2 \right] &= E\left[\sum_{i=1}^n Z_i^4\right] + 2 \sum_{i=1}^n\sum_{j=i+1}^n E[Z_i^2]E[Z_j^2]\\ &= 3n + n(n-1)\\ &= n^2+2n \end{align} giving $\displaystyle\operatorname{var}\left(\sum_{i=1}^n Z_i^2\right) = E\left[\left(\sum_{i=1}^n Z_i^2\right)^2 \right] - \left(E\left[\sum_{i=1}^n Z_i^2\right]\right)^2 = 2n$ as before.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! The second explanation is super straight-forward and easy to understand. $\endgroup$ – Nishant Kelkar Sep 15 '15 at 21:32
  • $\begingroup$ Just for clarity, in the second explanation, the -Inf/+Inf limits become (0/+Inf) because the function being integrated is an even function. $\endgroup$ – Nishant Kelkar Sep 15 '15 at 21:36
3
$\begingroup$

I was going to solve the integral with the chi squared density, but it seems that you want a derivation from the definition of a $\chi_n^2$ random variable as a sum of squares of $n$ independent standard normals (let's say $Z_i$). Working from this:

\begin{align} X^2 =& (\sum_{i=1}^nZ_i^2)^2\\ =& \sum_{i=1}^nZ_i^4+\sum_{i \neq j}^nZ_i^2Z_j^2 \end{align}

Therefore, using independence of $Z_i$s and linearity of expectation:

\begin{align} E(X^2)=& \sum_{i=1}^nE(Z_i^4)+\sum_{i \neq j}^nE(Z_i^2)E(Z_j^2)\\ =& \sum_{i=1}^nE(Z_i^4)+ n(n-1) \end{align}

Where we know that all the terms of the second sum are 1, and there are $n(n-1)$ terms. The only other thing that we need to know is the fourth moment (often called kurtosis) of the standard normal, which is 3. Since there are $n$ kurtosis terms, we have:

\begin{align} E(X^2)=& 3n + n(n - 1)\\ =& 2n+n^2 \end{align}

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ +1This demonstration is clear, to the point, and well-explained. Welcome to our site! $\endgroup$ – whuber Sep 15 '15 at 19:50
1
$\begingroup$

Thanks @Christopher Hanck for the hint!

My reference page: https://en.wikipedia.org/wiki/Moment-generating_function

In the table on the link above, the moment-generating function for the chi-squared distribution is given as: M(t) = (1 - 2t)^(-k/2)

Also, a very important section on the page linked to above, is the "Calculations of moments" section. This section gives the following formula:

E[X^n] = n'th derivative wrt. t of M(X, 0)

where M(X, 0) = (1 - 2t)^(-k/2) in our case

So basically, you take the derivative of the above function twice wrt. t, and then just plug in t = 0. And that's about it!

So after double differentiation wrt. t, you have,

E[X2^2] = k(k+2)(1 - 2t)^(-(k+4)/2)

Putting t = 0 above, we have

E[X2^2] = k(k + 2) = k^2 + 2k ... (for degree k)

From the question body above, we have:

var(X2) = E[(X2)^2] - (n^2)

= n^2 + 2n - n^2

= 2n

Q.E.D.

This does however, lead me to ask: How is the moment generating function of the chi-squared distribution derived?

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ take the definition of the chi^2-density $f(x;k)$ from en.wikipedia.org/wiki/Chi-squared_distribution and apply the definition of the mgf, $\int_0^\infty e^{tx}f(x;k)dx$. I do not remember having worked out the result myself, so I am not sure if the integration is easy to do by hand. $\endgroup$ – Christoph Hanck Sep 15 '15 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.