23
$\begingroup$

Using the student t-distribution with $k > 0$ degrees of freedom, location parameter $l$ and scale parameter $s$ having density

$$\frac{\Gamma \left(\frac{k+1}{2}\right)}{\Gamma\left(\frac{k}{2}\sqrt{k \pi s^2}\right)} \left\{ 1 + k^{-1}\left( \frac{x-l}{s}\right)\right\}^{-(k+1)/2},$$

how to show that the Student $t$-distribution can be written as a mixture of Gaussian distributions by letting $X\sim N(\mu,\sigma^2)$, $\tau = 1/\sigma^2\sim\Gamma(\alpha,\beta)$, and integrating the joint density $f(x,\tau|\mu)$ to get the marginal density $f(x|\mu)$? What are the parameters of the resulting $t$-distribution, as functions of $\mu,\alpha,\beta$?

I got lost in calculus by integrating the joint conditional density with the Gamma distribution.

$\endgroup$
31
$\begingroup$

The PDF of a Normal distribution is

$$f_{\mu, \sigma}(x) = \frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{(x-\mu )^2}{2 \sigma ^2}}dx$$

but in terms of $\tau = 1/\sigma^2$ it is

$$g_{\mu, \tau}(x) = \frac{\sqrt{\tau}}{\sqrt{2 \pi}} e^{-\frac{\tau(x-\mu )^2}{2 }}dx.$$

The PDF of a Gamma distribution is

$$h_{\alpha, \beta}(\tau) = \frac{1}{\Gamma(\alpha)}e^{-\frac{\tau}{\beta }} \tau^{-1+\alpha } \beta ^{-\alpha }d\tau.$$

Their product, slightly simplified with easy algebra, is therefore

$$f_{\mu, \alpha, \beta}(x,\tau) =\frac{1}{\beta^\alpha\Gamma(\alpha)\sqrt{2 \pi}} e^{-\tau\left(\frac{(x-\mu )^2}{2 } + \frac{1}{\beta}\right)} \tau^{-1/2+\alpha}d\tau dx.$$

Its inner part evidently has the form $\exp(-\text{constant}_1 \times \tau) \times \tau^{\text{constant}_2}d\tau$, making it a multiple of a Gamma function when integrated over the full range $\tau=0$ to $\tau=\infty$. That integral therefore is immediate (obtained by knowing the integral of a Gamma distribution is unity), giving the marginal distribution

$$f_{\mu, \alpha, \beta}(x) = \frac{\sqrt{\beta } \Gamma \left(\alpha +\frac{1}{2}\right) }{\sqrt{2\pi } \Gamma (\alpha )}\frac{1}{\left(\frac{\beta}{2} (x-\mu )^2+1\right)^{\alpha +\frac{1}{2}}}.$$

Trying to match the pattern provided for the $t$ distribution shows there is an error in the question: the PDF for the Student t distribution actually is proportional to

$$\frac{1}{\sqrt{k} s }\left(\frac{1}{1+k^{-1}\left(\frac{x-l}{s}\right)^2}\right)^{\frac{k+1}{2}}$$

(the power of $(x-l)/s$ is $2$, not $1$). Matching the terms indicates $k = 2 \alpha$, $l=\mu$, and $s = 1/\sqrt{\alpha\beta}$.


Notice that no Calculus was needed for this derivation: everything was a matter of looking up the formulas of the Normal and Gamma PDFs, carrying out some trivial algebraic manipulations involving products and powers, and matching patterns in algebraic expressions (in that order).

$\endgroup$
  • 10
    $\begingroup$ Inspired by this answer I made an animation of the t distribution as a mixture of normal distributions. It is available here: sumsar.net/blog/2013/12/t-as-a-mixture-of-normals $\endgroup$ – Rasmus Bååth Dec 6 '13 at 20:42
  • 1
    $\begingroup$ @whuber: Technically, for that kind of matching there is always an implicit use of calculus in your recognition that you can integrate out the gamma density using its known integral form. (This is the statistician's equivalent of hiding the broccoli by mixing it in with the meat and potatoes.) A clever way of hiding the calculus! $\endgroup$ – Reinstate Monica Jul 23 '18 at 3:28
1
$\begingroup$

I don't know the steps of the calculation, but I do know the results from some book (cannot remember which one...). I usually keep it in mind directly... :-) The Student $t$ distribution with $k$ degree freedom can be regarded as a Normal distribution with variance mixture $Y$, where $Y$ follows inverse gamma distribution. More precisely, $X$~$t(k)$,$X$=$\sqrt Y$*$\Phi$,where $Y$~$IG(k/2,k/2)$,$\Phi$ is standard normal rv. I hope this could help you in some sense.

$\endgroup$
0
$\begingroup$

To simplify we assume mean $0$. Using representation, we show the result for integer degrees of freedom. $$ \sqrt{1/\tau} X =Y $$ is equivalent to a Gaussian mixture with that prior: conditioned on $\tau$, $Y$ is Gaussian with precision $\tau$, and the prior $\tau$ is as desired. Then it remains to show that $\sqrt{1/\tau} X$ is a t-distribution. We can write $$ \tau \sim \Gamma(\alpha, \beta) \sim \frac{\beta}{2} \Gamma(\alpha, 2) \sim \frac{\beta}{2} \chi^2(2\alpha) $$ using a well-known result about gammas and Chi-squares (decompose a gamma as a sum of exponentials and combine the exponentials to normals to Chi squares) This in turn implies that $$ Y \sim X \frac{1}{\sqrt{(\beta/2) \chi^2(2\alpha)} } $$ $$= \frac{ X \sqrt{\alpha \beta} }{\sqrt{ \chi^2_{2\alpha}/(2\alpha)}} $$ which is a scaled t with $ k=2\alpha$ and $s=1/\sqrt{\alpha \beta} $ by variance of t. We can recenter our representation at $\mu$ and $l$ would follow.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.