22
$\begingroup$

Let $t_i$ be drawn i.i.d from a Student t distribution with $n$ degrees of freedom, for moderately sized $n$ (say less than 100). Define $$T = \sum_{1\le i \le k} t_i^2$$ Is $T$ distributed nearly as a chi-square with $k$ degrees of freedom? Is there something like the Central Limit Theorem for the sum of squared random variables?

$\endgroup$
  • $\begingroup$ @suncoolsu : it does say 'nearly'... $\endgroup$ – shabbychef Jan 17 '11 at 7:16
  • $\begingroup$ my apologies. didn't see that. $\endgroup$ – suncoolsu Jan 17 '11 at 15:18
15
$\begingroup$

Answering the first question.

We could start from the fact noted by mpiktas, that $t^2 \sim F(1, n)$. And then try a more simple step at first - search for the distribution of a sum of two random variables distributed by $F(1,n)$. This could be done either by calculating the convolution of two random variables, or calculating the product of their characteristic functions.

The article by P.C.B. Phillips shows that my first guess about "[confluent] hypergeometric functions involved" was indeed true. It means that the solution will be not trivial, and the brute-force is complicated, but necessary condition to answer your question. So since $n$ is fixed and you sum up t-distributions, we can't say for sure what the final result will be. Unless someone has a good skill playing with products of confluent hypergeometric functions.

$\endgroup$
  • 2
    $\begingroup$ +1 for the link, did not know that characteristic function of F distribution was so complicated. $\endgroup$ – mpiktas Jan 17 '11 at 11:03
16
$\begingroup$

It's not even a close approximation. For small $n$, the expectation of $T$ equals $\frac{k n}{n-2}$ whereas the expectation of $\chi^2(k)$ equals $k$. When $k$ is small (less than 10, say) histograms of $\log(T)$ and of $\log(\chi^2(k))$ don't even have the same shape, indicating that shifting and rescaling $T$ still won't work.

Intuitively, for small degrees of freedom Student's $t$ is heavy tailed. Squaring it emphasizes that heaviness. The sums therefore will be more skewed--usually much more skewed--than sums of squared normals (the $\chi^2$ distribution). Calculations and simulations bear this out.


Illustration (as requested)

alt text

Each histogram depicts an independent simulation of 100,000 trials with the specified degrees of freedom ($n$) and summands ($k$), standardized as described by @mpiktas. The value of $n=9999$ on the bottom row approximates the $\chi^2$ case. Thus you can compare $T$ to $\chi^2$ by scanning down each column.

Note that standardization is not possible for $n \lt 5$ because the appropriate moments do not even exist. The lack of stability of shape (as you scan from left to right across any row or from top to bottom down any column) is even more marked for $n \le 4$.


Finally, let's address the question about a central limit theorem. Since the square of a random variable is a random variable, the usual Central Limit Theorem automatically applies to sequences of independent squared random variables like those in the question. For its conclusion (convergence of the standardized sum to Normality) to hold, we need the squared random variable to have finite variance.

Which Student t variables, when squared, have finite variance? When $X$ is any random variable, by one standard definition the variance of its square $Y=X^2$ is

$$\operatorname{Var}(Y) = E[Y^2] - E[Y]^2 = E[X^4] - E[X^2]^2.$$

Finiteness of $E[X^4]$ will assure finiteness of $E[X^2].$ Because the student $t$ density with $\nu$ degrees of freedom is (up to a rescaling of $X$) proportional to $f_{\nu}(x)=(1+x^2)^{-(\nu+1)/2},$ the question comes down to the finiteness of the integral of $x^4$ times this. Because the product is bounded, we are concerned with the behavior as $|x|\to\infty,$ where the integrand is asymptotically

$$x^4 f_{\nu}(x) \sim x^4 (x^2)^{-(\nu+1)/2} = x^{3 - \nu}.$$

Its integral diverges when the exponent exceeds $-1$ and otherwise converges; that is,

The standardized version of $T$ converges to a standard Normal distribution if and only if $\nu \gt 4.$

$\endgroup$
  • $\begingroup$ I was afraid of that, but I thought that summing would bring in the tails somewhat. $\endgroup$ – shabbychef Jan 17 '11 at 20:53
  • $\begingroup$ I also thought to produce some sort of Monte Carlo experiments, trying to see for what $n$ and $k$ the approximation could be close enough to $\chi^2(k)$, probably $k(n)$ that we need here. But for small $k$ and especially $n$ it will be very heavy tailed indeed. May be you could add here these two histograms, just for lazy people like me? $\endgroup$ – Dmitrij Celov Jan 17 '11 at 21:22
  • $\begingroup$ @Dmitrij The simulations are fast (it takes more time to draw the histograms), so I added 12 of them. $\endgroup$ – whuber Jan 17 '11 at 22:00
  • $\begingroup$ +1 for the figure. Illustrations are always nice to see. $\endgroup$ – Dmitrij Celov Jan 18 '11 at 17:16
8
$\begingroup$

I'll answer second question. The central limit theorem is for any iid sequence, squared or not squared. So in your case if $k$ is sufficiently large we have

$\dfrac{T-kE(t_1)^2}{\sqrt{kVar(t_1^2)}}\sim N(0,1)$

where $Et_1^2$ and $Var(t_1^2)$ is respectively the mean and variance of squared Student t distribution with $n$ degrees of freedom. Note that $t_1^2$ is distributed as F distribution with $1$ and $n$ degrees of freedom. So we can grab the formulas for mean and variance from wikipedia page. The final result then is:

$\dfrac{T-k\frac{n}{n-2}}{\sqrt{k\frac{2n^2(n-1)}{(n-2)^2(n-4)}}}\sim N(0,1)$

$\endgroup$
  • 1
    $\begingroup$ Hotelling's T^2: (f − d + 1)/fd T^2 ∼ F (d , f + 1 − d ) $\endgroup$ – DWin Jan 17 '11 at 8:04
  • 1
    $\begingroup$ @DWin, not so sure that Hotelling's $T^2$ is really applicable here. At least from formulas from wikipedia page it is not immediately clear that $T$ in OP question can be represented as $T^2$. Can you please elaborate on this more? $\endgroup$ – mpiktas Jan 17 '11 at 8:25
  • $\begingroup$ will search for a convolution of $F(1,n)+F(1,n)$, afraid of some hypergeometric things, but have to be known somewhere. $\endgroup$ – Dmitrij Celov Jan 17 '11 at 9:41
  • $\begingroup$ I believe it reduces to your situation when the variance matrix is diagonal. The off-diagonal elements from a sample should be near zero if the samples were from Normal, but might not be exactly zero if from t. Nonetheless, you asked for something approximate, so I think the answer is probably F under that proviso. $\endgroup$ – DWin Jan 17 '11 at 17:55
  • $\begingroup$ @DWin: it sure does look like a Hotelling with diagonal covariance matrix, but I am somewhat confused: from first principles, it does not seem like the sum of $F(1,n)$ RVs would be distributed like an $F$... $\endgroup$ – shabbychef Jan 17 '11 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.