8
$\begingroup$

I am having trouble finding a resource online that derives the expected Fisher's Information Matrix for the uni-variate Student's t-distribution. Does anyone know of such a resource?

In absence of any existing resource that derives the expected Fisher's information matrix for the t-distribution, I am trying to derive it myself but I'm stuck. Here is my work so far:

$y_i \sim t(\mu, \sigma^2, v)$ where $v$ is the degrees of freedom (df) parameter (assumed fixed). Then: \begin{align*} f(y_i) &= \frac{\Gamma(\frac{v+1}{2})}{\Gamma(\frac{v}{2})\sqrt{\pi v \sigma^2}}\big(1+\frac{1}{v\sigma^2}(y_i-\mu)^2\big)^{\frac{-(v+1)}{2}} \end{align*}

Thus we have the following log-likelihood function: \begin{align*} log f(y_i)=log\Gamma(\frac{v+1}{2})-log\Gamma(\frac{v}{2})-\frac{1}{2}log(\pi v \sigma^2)+ \frac{-(v+1)}{2}log\big[1+\frac{1}{v\sigma^2}(y_i-\mu)^2\big] \end{align*}

Here the first derivative equations:

\begin{align*} &\frac{\partial}{\partial \mu}logf(y_i)=\frac{v+1}{2}\frac{\frac{2}{v\sigma^2}(y_i-\mu)}{1+\frac{1}{v\sigma^2}(y_i-\mu)^2} \\ & \frac{\partial}{\partial \sigma^2}logf(y_i)= \frac{-1}{2\sigma^2}-\frac{(v+1)}{2} \frac{\frac{-1}{v\sigma^4}(y_i-\mu)^2}{1+\frac{1}{v\sigma^2}(y_i-\mu)^2} \end{align*}

And here are the 2nd derivative equations:

\begin{align*} &\frac{\partial}{\partial \mu^2}logf(y_i)=\frac{v+1}{2}\frac{\frac{-2}{v\sigma^2}+\frac{2}{dv^2\sigma^4}(y_i-\mu)^2}{\big(1+\frac{1}{v\sigma^2}(y_i-\mu)^2\big)^2} \\ & \frac{\partial}{\partial \mu \partial \sigma^2}logf(y_i) =\frac{v+1}{2} \Big\{[\frac{2}{v\sigma^2}-\frac{4}{v^2\sigma^6}(y_i-\mu)^2][1+\frac{1}{v\sigma^2}(y_i-\mu)^2]^2-[\frac{-2}{v\sigma^2}+\frac{2}{v^2\sigma^4}(y_i-\mu)^2]*2[1+\frac{1}{v\sigma^2}(y_i-\mu)^2][\frac{-1}{v\sigma^4}(y_i-\mu)^2]\Big\}/\Big\{ [1+\frac{1}{v\sigma^2}(y_i-\mu)^2]^4\Big\}.....\text{really messy!} \\ &\frac{\partial}{\partial (\sigma^2)^2}logf(y_i)=\frac{1}{2\sigma^4}-\frac{(v+1)}{2}\frac{\frac{1}{v\sigma^6}(y_i-\mu)^2}{[1+\frac{1}{v\sigma^2}(y_i-\mu)^2]^2} \end{align*}

Finally, the expected fisher's information matrix is calculated as follows: \begin{align*} \boldsymbol{I}= -\mathbb{E}\Big(\begin{bmatrix} \frac{\partial^2}{\partial \mu^2}logf(y_i) & \frac{\partial}{\partial \mu \partial \sigma^2}logf(y_i) \\ \frac{\partial}{\partial \mu \partial \sigma^2}logf(y_i) & \frac{\partial^2}{\partial (\sigma^2)^2}logf(y_i) \end{bmatrix}\Big) \end{align*}

However, I have no idea how to calculate these expectations. Is anyone aware of a resource that has done this? Honestly, the only quantity I'm interested in is: $-\mathbb{E}\big[\frac{\partial^2}{\partial (\sigma^2)^2}logf(y_i)\big]$, would someone at least be able to help me calculate this?

$\endgroup$
3
$\begingroup$

It was brought to my attention that Lange et al 1989 derived the expected Fisher's Information for the multivariate t-distribution in Appendix B. Therefore, I got the answer I wanted, you can regard this question as answered!

In particular, using the result of Lange et al, I derived the following Fisher's Information Matrix for the univariate t-distribution (with fixed degrees of freedom parameter $v$):

\begin{align*} \boldsymbol{I}=\begin{bmatrix} \frac{v+1}{(v+3)\sigma^2} & 0 \\ 0 & \frac{v}{2(v+3)\sigma^4} \end{bmatrix} \end{align*}

$\endgroup$
  • 1
    $\begingroup$ Is there any reference where the Fisher's Information Matrix has been derived for variable degrees of freedom parameter, i.e. 3-Dimensional Fisher's Information Matrix where the scale, location and degrees of freedom are all provided ? $\endgroup$ – uday Sep 30 '17 at 9:18
  • 1
    $\begingroup$ I have the same question. Do we have a 3x3 Fisher matrix that includes the nu parameter? $\endgroup$ – Riemann1337 Oct 27 '17 at 5:17
  • $\begingroup$ Above result confirmed correct with FisherInformation function in mathStatica $\endgroup$ – wolfies Dec 1 '17 at 18:49
1
$\begingroup$

It is not difficult (but a bit tedious) by using the formula $$ \mathcal{I}(\mu, \sigma^2) = \mathbb{E}\left[\begin{pmatrix}{\left(\frac{\partial}{\partial\mu} \log f(Y)\right)}^2 & \left(\frac{\partial}{\partial\mu} \log f(Y)\right)\left(\frac{\partial}{\partial\sigma} \log f(Y)\right) \\ \left(\frac{\partial}{\partial\mu} \log f(Y)\right)\left(\frac{\partial}{\partial\sigma} \log f(Y)\right) & {\left(\frac{\partial}{\partial\sigma^2} \log f(Y)\right)}^2 \end{pmatrix} \right]. $$ First, observe that by the change of variables $y \mapsto y-\mu$ in any involved integral, one can take $\mu=0$ in the calculations.

The calculations rely on the following integral: $$ I(\lambda, a, b) := \int_0^\infty y^{2a-1}\left(1+\frac{1}{\lambda}y^2\right)^{-\frac{2a+b}{2}} \textrm{d}y = \frac{\lambda^a}{2} B\left(a, \frac{b}{2}\right). $$ This equality is obtained by the change of variables $y \mapsto y^2$ and with the help of the density of the Beta prime distribution.

Observe that the integrand is an even function when $2a-1$ is an even integer, hence $$ J(\lambda, a, b) := \int_{-\infty}^{+\infty} y^{2a-1}\left(1+\frac{1}{\lambda}y^2\right)^{-\frac{p+1+b}{2}} \textrm{d}y = 2 I(\lambda, a, b) = \lambda^a B\left(a, \frac{b}{2}\right). $$

I will detail only the first calculation. Set $$ K(\nu, \sigma)=\frac{1}{B\left(\frac12,\frac{\nu}{2}\right)}\frac{1}{\sqrt{\nu \sigma^2}}, $$ the normalization constant of the density.

One has $$ \begin{align} \mathbb{E}\left[{\left(\frac{\partial}{\partial\mu} \log f(Y)\right)}^2 \right] = K(\nu, \sigma){\left(\frac{\nu+1}{\nu\sigma^2}\right)}^2 J\left(\nu\sigma^2, \frac{3}{2}, \nu+2\right). \end{align} $$ Since $\frac{B\left(\frac12,\frac{\nu}{2}\right)}{B\left(\frac{3}{2},\frac{\nu+2}{2}\right)} = \frac{B\left(\frac12,\frac{\nu}{2}\right)}{B\left(\frac{3}{2},\frac{\nu}{2}\right)}\frac{B\left(\frac{3}{2},\frac{\nu}{2}\right)}{B\left(\frac{3}{2},\frac{\nu+2}{2}\right)} = \frac{(\nu+1)}{1}\frac{(\nu+3)}{\nu}$, we find $$ \mathbb{E}\left[{\left(\frac{\partial}{\partial\mu} \log f(Y)\right)}^2 \right] = \frac{\nu}{\nu+3}(\nu+1){(\nu\sigma^2)}^{-1/2-2+3/2} = \frac{\nu+1}{(\nu+3)\sigma^2}. $$ The second calculation is easy: $$ \mathbb{E}\left[ \left(\frac{\partial}{\partial\mu} \log f(Y)\right)\left(\frac{\partial}{\partial\sigma} \log f(Y)\right)\right] = 0 $$ because it only involves integrals of odd functions.

Finally the calculation of $$ \mathbb{E}\left[{\left(\frac{\partial}{\partial\sigma^2} \log f(Y)\right)}^2 \right] $$ is the more tedious and I skip it. Its calculation involves integrals $J(\nu\sigma^2, a, b)$ with $2a-1$ even integer, whose value is given above.

I've done the calculations and I've found $$ \frac{{(\nu+1)}^2}{4{(\nu\sigma^4)}^2} K(\nu, \sigma^2) J\left(\nu\sigma^2, \frac{5}{2},\nu\right) - \frac{\nu+1}{2\nu\sigma^6} K(\nu, \sigma^2) J\left(\nu\sigma^2, \frac{3}{2},\nu\right) + \frac{1}{4\sigma^4} $$ and this simplifies to $$ \frac{\nu}{2(\nu+3)\sigma^4}. $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.