# What is the Fisher's information matrix for the Wishart distribution?

I have been struggling computing the Fisher's information of the Wishart distribution. I'll write what I have gone through. Let's $\Omega$ denote a $p\times p$ Wishart random variate denoted by $\mathcal{W}(k,V)$ where $k$ is the degrees of freedom and $V$ a positive definite scale matrix. If we write $\mathcal{W}(\Omega\,|\,k,V)$ for the density function, \begin{align} \nabla_{\operatorname{vech}(V)}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}D_{p}'\left(V^{-1}\otimes V^{-1}\right)D_{p}\operatorname{vech}(\Omega)-\dfrac{k}{2}D_{p}\operatorname{vech}\left(V^{-1}\right)\\ \nabla_{k}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}\log|\Omega|-\dfrac{1}{2}\log|V|-\dfrac{p}{2}\log 2-\dfrac{1}{2}\sum_{i=1}^{p}\psi\left(\dfrac{k+1-i}{2}\right) \end{align} where $D_{p}$ is the unique duplication matrix such that $D_{p}\operatorname{vech}(A)=\operatorname{vec}(A)$, $\otimes$ Kronecker product, and $\psi$ digamma function.

Is there a closed-form expression for the Fisher information matrix of the Wishart distribution?

UPDATE I have computed the following two terms: \begin{align}\operatorname{Var}(\operatorname{vec}(\Omega)) &= k\left(\mathbf{I}_{p^{2}}+K_{pp}\right)(V\otimes V)\\ \operatorname{Var}(\log|\Omega|) &= \sum_{i=1}^{p}\psi_{1}\left(\dfrac{k-i+1}{2}\right) \end{align} where $K_{pp}$ is a $p^{2}\times p^{2}$ commutation matrix such that $$$$K_{pp}\operatorname{vec}(C) = \operatorname{vec}(C')$$$$ for a $p\times p$ matrix $C$ and $\psi_{1}$ is the trigamma function. But I have no idea how to get the following covariance term. $$\operatorname{Cov}(\operatorname{vec}(\Omega),\log|\Omega|)$$

• self-study? Cross-post? – user603 Oct 8 '17 at 10:17


I've derived it with the second order differential.

The log-likelihood is \begin{align} \ell(\nu, \Sigma \mid W) & = C - \frac{\nu p}{2} \log 2 - \log\Gamma_p\left(\frac{\nu}{2}\right) - \frac{\nu}{2} \log |\Sigma| \\ & + \frac{\nu-p-1}{2} \log |W| -\frac{1}{2} \tr(\Sigma^{-1}W). \end{align} Its differential at $$(\nu_0, \Sigma_0)$$ is \begin{align} \D_{\nu_0,\Sigma_0}\ell & = 0 - \frac{p}{2} \log 2\,\D\nu - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu - \frac{1}{2}\log |\Sigma_0|\,\D\nu - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \\ & + \frac{1}{2}\log |W|\,\D\nu + \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr). \end{align} Let's differentiate each term. $$\D_{\nu_0,\Sigma_0}\left\{ - \frac{p}{2} \log 2\,\D\nu \right\} = 0.$$ $$\D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu \right\} = - \frac{1}{4}\psi'_p\left(\frac{\nu_0}{2}\right)\D\nu\D\nu$$ \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\log |\Sigma_0|\,\D\nu \right\} & = - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \right\} & = \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = \frac{\nu_0}{2} {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma) - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} $$\D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2}\log |W|\,\D\nu \right\} = 0.$$ $$\D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2} \tr(\Sigma_0^{-1}\D\Sigma\Sigma_0^{-1}W) \right\} = - 2 \times \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr).$$ Since $$\mathbb{E}[W] = \nu_0\Sigma_0$$, the expectation of the term above (which is the only term depending on $$W$$) is $$- \nu_0\tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr) = - \nu_0 {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma).$$ Finally, note that $$\vec(\D\Sigma) = D_p \vech(\D\Sigma),$$ where $$D_p$$ denotes the duplication matix, then the Fisher information matrix for the parameterization $$\bigl(\nu, \text{vech}(\Sigma)\bigr)$$ is $${\cal I}(\nu, \Sigma) = \begin{pmatrix} \frac{1}{4}\psi'_p\left(\frac{\nu}{2}\right) & \frac{1}{2}{\bigl(D_p' \vec(\Sigma^{-1})\bigr)}' \\ \frac{1}{2} D_p' \vec(\Sigma^{-1}) & \frac{\nu}{2} D_p'(\Sigma_0^{-1} \otimes \Sigma_0^{-1})D_p \end{pmatrix}.$$ Let's check:

psi1p <- function(p, a){ # second derivative of log Gamma_p
sum(gsl::psi_1(a + (1-(1:p))/2))
}
p <- 2
Dp <- matrixcalc::duplication.matrix(p)
logL <- function(params, W){ # log likelihood
Sigma <- matrix(Dp %*% params[-1], p, p)
LaplacesDemon::dwishart(W, params[1], Sigma, log=TRUE)
}
# parameters
nu <- 6
Sigma <- toeplitz(p:1)
# Information matrix obtained by simulations
nsims <- 10000
Wsims <- matrixsampling::rwishart(nsims, nu, Sigma)
Hsims <- array(NA_real_, dim=c(4,4,nsims))
for(i in 1:nsims){
Hsims[,,i] <- numDeriv:: hessian(logL,
c(nu, Sigma[upper.tri(Sigma, diag=TRUE)]),
W=Wsims[,,i])
}
round(apply(-Hsims, 1:2, mean), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.32 -1.31  0.32
## [3,] -0.33 -1.31  3.28 -1.30
## [4,]  0.33  0.32 -1.30  1.31
# Exact information matrix
I11 <- psi1p(p, nu/2)/4
I21 <- t(Dp) %*% c(solve(Sigma)) / 2
I22 <- nu/2 * t(Dp) %*% (kronecker(solve(Sigma), solve(Sigma))) %*% Dp
round(rbind(c(I11, I21), cbind(I21, I22)), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.33 -1.33  0.33
## [3,] -0.33 -1.33  3.33 -1.33
## [4,]  0.33  0.33 -1.33  1.33