15
$\begingroup$

let $Z \sim N(0,1)$ and $W \sim \chi^2(s)$.

If $Z$ and $W$ are independently distributed then the variable $Y = \frac{Z}{\sqrt{W/s}}$ follows a $t$ distribution with degrees of freedom $s$.

I am looking for a proof of this fact, a reference is good enough if you do not want to write down the complete argument.

$\endgroup$
2
  • 1
    $\begingroup$ This is demonstrated formally at stats.stackexchange.com/questions/52906: the ratio, when written as an integral, is seen to be a mixture of Gaussians, and that demonstration shows that the mixture is a t distribution. $\endgroup$
    – whuber
    May 13, 2015 at 13:22
  • 2
    $\begingroup$ In some textbooks this is a definition of a t-distribution. You do not need to prove it. How to derive a pdf given such a definition is however a valid question. $\endgroup$
    – mpiktas
    May 15, 2015 at 8:04

3 Answers 3

16
$\begingroup$

Let $Y$ be a chi-square random variable with $n$ degrees of freedom. Then the square-root of $Y$, $\sqrt Y\equiv \hat Y$ is distributed as a chi-distribution with $n$ degrees of freedom, which has density $$ f_{\hat Y}(\hat y) = \frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} \hat y^{n-1} \exp\Big \{{-\frac {\hat y^2}{2}} \Big\} \tag{1}$$

Define $X \equiv \frac {1}{\sqrt n}\hat Y$. Then $ \frac {\partial \hat Y}{\partial X} = \sqrt n$, and by the change-of-variable formula we have that

$$ f_{X}(x) = f_{\hat Y}(\sqrt nx)\Big |\frac {\partial \hat Y}{\partial X} \Big| = \frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} (\sqrt nx)^{n-1} \exp\Big \{{-\frac {(\sqrt nx)^2}{2}} \Big\}\sqrt n $$

$$=\frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} n^{\frac n2}x^{n-1} \exp\Big \{{-\frac {n}{2}x^2} \Big\} \tag{2}$$

Let $Z$ be a standard normal random variable, independent from the previous ones, and define the random variable

$$T = \frac{Z}{\sqrt \frac Yn}= \frac ZX $$.

By the standard formula for the density function of the ratio of two independent random variables, $$f_T(t) = \int_{-\infty}^{\infty} |x|f_Z(xt)f_X(x)dx $$

But $f_X(x) = 0$ for the interval $[-\infty, 0]$ because $X$ is a non-negative r.v. So we can eliminate the absolute value, and reduce the integral to

$$f_T(t) = \int_{0}^{\infty} xf_Z(xt)f_X(x)dx $$

$$ = \int_{0}^{\infty} x \frac{1}{\sqrt{2\pi}}\exp \Big \{{-\frac{(xt)^2}{2}}\Big\}\frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} n^{\frac n2}x^{n-1} \exp\Big \{{-\frac {n}{2}x^2} \Big\}dx $$

$$ = \frac{1}{\sqrt{2\pi}}\frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} n^{\frac n2}\int_{0}^{\infty} x^n \exp \Big \{-\frac 12 (n+t^2) x^2\Big\} dx \tag{3}$$

The integrand in $(3)$ looks promising to eventually be transformed into a Gamma density function. The limits of integration are correct, so we need to manipulate the integrand into becoming a Gamma density function without changing the limits. Define the variable

$$m \equiv x^2 \Rightarrow dm = 2xdx \Rightarrow dx = \frac {dm}{2x}, \; x = m^{\frac 12}$$ Making the substitution in the integrand we have

$$I_3=\int_{0}^{\infty} x^n \exp \Big \{-\frac 12 (n+t^2) m\Big\} \frac {dm}{2x} \\ = \frac 12\int_{0}^{\infty} m^{\frac {n-1}{2}} \exp \Big \{-\frac 12 (n+t^2) m\Big \} dm \tag{4}$$

The Gamma density can be written

$$ Gamma(m;k,\theta) = \frac {m^{k-1} \exp\Big\{-\frac{m}{\theta}\Big \}}{\theta^k\Gamma(k)}$$

Matching coefficients, we must have

$$k-1 = \frac {n-1}{2} \Rightarrow k^* = \frac {n+1}{2}, \qquad \frac 1\theta =\frac 12 (n+t^2) \Rightarrow \theta^* = \frac 2 {(n+t^2)} $$

For these values of $k^*$ and $\theta^*$ the terms in the integrand involving the variable are the kernel of a gamma density. So if we divide the integrand by $(\theta^*)^{k^*}\Gamma(k^*)$ and multiply outside the integral by the same magnitude, the integral will be the gamma distr. function and will equal unity. Therefore we have arrived at

$$I_3 = \frac12(\theta^*)^{k^*}\Gamma(k^*) = \frac12 \Big (\frac 2 {n+t^2}\Big ) ^{\frac {n+1}{2}}\Gamma\left(\frac {n+1}{2}\right) = 2^ {\frac {n-1}{2}}n^{-\frac {n+1}{2}}\Gamma\left(\frac {n+1}{2}\right)\left(1+\frac {t^2}{n}\right)^{-\frac 12 (n+1)} $$

Inserting the above into eq. $(3)$ we get

$$f_T(t) = \frac{1}{\sqrt{2\pi}}\frac {2^{1-\frac n2}}{\Gamma\left(\frac {n}{2}\right)} n^{\frac n2}2^ {\frac {n-1}{2}}n^{-\frac {n+1}{2}}\Gamma\left(\frac {n+1}{2}\right)\left(1+\frac {t^2}{n}\right)^{-\frac 12 (n+1)}$$

$$=\frac{\Gamma[(n+1)/2]}{\sqrt{n\pi}\,\Gamma(n/2)}\left(1+\frac {t^2}{n}\right)^{-\frac 12 (n+1)}$$

...which is what is called the (density function of) the Student's t-distribution, with $n$ degrees of freedom.

$\endgroup$
32
$\begingroup$

Although E. S. Pearson didn't like it, Fisher's original argument was geometric, simple, convincing, and rigorous. It relies on a small number of intuitive and easily established facts. They are easily visualized when $s=1$ or $s=2$, where the geometry can be visualized in two or three dimensions. In effect, it amounts to using cylindrical coordinates in $\mathbb{R}^s\times\mathbb{R}$ to analyze $s+1$ iid Normal variables.

  1. $s+1$ independent and identically distributed Normal variates $X_1, \ldots, X_{s+1}$ are spherically symmetrical. This means that the radial projection of the point $(X_1, \ldots, X_{s+1})$ onto the unit sphere $S^s \subset \mathbb{R}^{s+1}$ has a uniform distribution on $S^s$.

  2. A $\chi^2(s)$ distribution is that of the sum of squares of $s$ independent standard Normal variates.

  3. Thus, setting $Z=X_{s+1}$ and $W = X_1^2 + \cdots + X_s^2$, the ratio $Z/\sqrt{W}$ is the tangent of the latitude $\theta$ of the point $(X_1, \ldots, X_s, X_{s+1})$ in $\mathbb{R}^{s+1}$.

  4. $\tan\theta$ is unchanged by radial projection onto $S^s$.

  5. The set determined by all points of latitude $\theta$ on $S^s$ is an $s-1$ dimensional sphere of radius $\cos \theta$. Its $s-1$ dimensional measure therefore is proportional to $$\cos^{s-1}\theta = (1 + \tan^2\theta)^{-(s-1)/2}.$$

  6. The differential element is $\mathrm{d}(\tan\theta) = \cos^{-2}\theta \,\mathrm{d}\theta = (1 + \tan^2\theta) \,\mathrm{d}\theta$.

  7. Writing $t = Z/\sqrt{W/s} = \sqrt{s}\tan\theta$ gives $\tan\theta = t/\sqrt{s}$, whence $$1+t^2/s = 1+\tan^2\theta$$ and $$\mathrm{d}t = \sqrt{s}\,\mathrm{d}\tan\theta = \sqrt{s}(1+\tan^2\theta)\,\mathrm{d}\theta.$$ Together these equations imply $$\mathrm{d}\theta = \frac{1}{\sqrt{s}} \left(1+t^2/s\right)^{-1}\mathrm{d}t.$$ Incorporating the factor of $1/\sqrt{s}$ into a normalizing constant $C(s)$ shows the density of $t$ is proportional to

    $$(1 + \tan^2\theta)^{-(s-1)/2}\,\mathrm{d}\theta = (1 + t^2/s)^{-(s-1)/2}\ (1 + t^2/s)^{-1}\,\mathrm{d}t = (1 + t^2/s)^{-(s+1)/2}\,\mathrm{d}t.$$

That is the Student t density.

Figure

The figure depicts the upper hemisphere (with $Z \ge 0$) of $S^s$ in $\mathbb{R}^{s+1}$. The crossed axes span the $W$-hyperplane. The black dots are part of a random sample of a $s+1$-variate standard Normal distribution: they are the values projecting to a constant given latitude $\theta$, shown as the yellow band. The density of these dots is proportional to the $s-1$-dimensional volume of that band, which itself is an $S^{s-1}$ of radius $\theta$. The cone over that band is drawn to terminate at a height of $\tan \theta$. Up to a factor of $\sqrt{s}$, the Student t distribution with $s$ degrees of freedom is the distribution of this height as weighted by the measure of the yellow band upon normalizing the area of the unit sphere $S^s$ to unity.

Incidentally, the normalizing constant must be $1/\sqrt{s}$ (as previously mentioned) times the relative volumes of the spheres,

$$\eqalign{ C(s) &= \frac{1}{\sqrt{s}} \frac{|S^{s-1}|}{|S^s|} = \frac{1}{\sqrt{s}} \frac{s \pi^{s/2} \Gamma(\frac{s+1}{2} + 1)}{(s+1)\pi^{(s+1)/2} \Gamma(\frac{s}{2}+1)} \\ &=\frac{1}{\sqrt{s}} \frac{s \pi^{s/2} (s+1)/2\Gamma(\frac{s+1}{2})}{(s+1)\pi^{(s+1)/2} (s/2)\Gamma(\frac{s}{2})} \\ &= \frac{\Gamma(\frac{s+1}{2})}{\sqrt{s\pi}\Gamma(\frac{s}{2})}. }$$

The final expression, although conventional, slightly disguises the beautifully simple initial expression, which clearly reveals the meaning of $C(s)$.


Fisher explained this derivation to W. S. Gosset (the original "Student") in a letter. Gosset attempted to publish it, giving Fisher full credit, but Pearson rejected the paper. Fisher's method, as applied to the substantially similar but more difficult problem of finding the distribution of a sample correlation coefficient, was eventually published.

References

R. A. Fisher, Frequency Distribution of the Values of the Correlation Coefficient in Samples from an Indefinitely Large Population. Biometrika Vol. 10, No. 4 (May, 1915), pp. 507-521. Available on the Web at https://stat.duke.edu/courses/Spring05/sta215/lec/Fish1915.pdf (and at many other places via searching, once this link disappears).

Joan Fisher Box, Gosset, Fisher, and the t Distribution. The American Statistician, Vol. 35, No. 2 (May, 1981), pp. 61-66. Available on the Web at http://social.rollins.edu/wpsites/bio342spr13/files/2015/03/Studentttest.pdf.

E.L. Lehmann, Fisher, Neyman, and the Creation of Classical Statistics. Springer (2011), Chapter 2.

$\endgroup$
1
  • 1
    $\begingroup$ That's something noble, here. +1. Normally I don't dabble in geometric perspective but that's an ingenious take. $\endgroup$ Aug 25, 2022 at 19:22
1
$\begingroup$

I would try change of variables. Set $Y=\frac{Z}{\sqrt{\frac{W}{s}}}$ and $X=Z$ for example. So $Z=X$, $W=\frac{sX^2}{Y^2}$. Then $f_{X,Y}(x,y)=f_{Z,W}(x,\frac{sx^2}{y^2})|\det(J)|$. Where $J$ is the Jacobian matrix for the multivariate function of $Z$ and $W$ of $X$ and $Y$. Then you can integrate $x$ out from the joint density. $\frac{\partial Z}{\partial X}=1$, $\frac{\partial Z}{\partial Y}=0$, $\frac{\partial W}{\partial X}=\frac{2sX}{Y^2}$, and $\frac{\partial W}{\partial Y}=\frac{-2sX^2}{Y^3}$.

$$ J= \begin{pmatrix} 1&0\\ *&\frac{-2sX^2}{Y^3} \end{pmatrix} $$

So $|\det(J)|=\frac{2sx^2}{y^3}$. I just took a look at Elements of Distribution Theory by Thomas A. Severini and there, they take $X=W$. Integrating things out becomes easier using properties of a Gaama distribution. If I use $X=Z$, I probably would need to complete squares.

But I don't want to do the calculation.

$\endgroup$
2
  • 1
    $\begingroup$ I did not downvote you, in fact I just upvoted you. But I think maybe the downvote arrived before your edit. $\endgroup$
    – Monolite
    May 12, 2015 at 1:58
  • $\begingroup$ Sorry about that, I will be careful from now on. $\endgroup$
    – ztyh
    May 12, 2015 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.