What is the Fisher's information matrix for the Wishart distribution?

Question

I have been struggling computing the Fisher's information of the Wishart distribution. I'll write what I have gone through. Let's $\Omega$ denote a $p\times p$ Wishart random variate denoted by $\mathcal{W}(k,V)$ where $k$ is the degrees of freedom and $V$ a positive definite scale matrix. If we write $\mathcal{W}(\Omega\,|\,k,V)$ for the density function, $$ \begin{align} \nabla_{\operatorname{vech}(V)}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}D_{p}'\left(V^{-1}\otimes V^{-1}\right)D_{p}\operatorname{vech}(\Omega)-\dfrac{k}{2}D_{p}\operatorname{vech}\left(V^{-1}\right)\\ \nabla_{k}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}\log|\Omega|-\dfrac{1}{2}\log|V|-\dfrac{p}{2}\log 2-\dfrac{1}{2}\sum_{i=1}^{p}\psi\left(\dfrac{k+1-i}{2}\right) \end{align} $$ where $D_{p}$ is the unique duplication matrix such that $D_{p}\operatorname{vech}(A)=\operatorname{vec}(A)$, $\otimes$ Kronecker product, and $\psi$ digamma function.

Is there a closed-form expression for the Fisher information matrix of the Wishart distribution?

---

UPDATE I have computed the following two terms: $$ \begin{align}\operatorname{Var}(\operatorname{vec}(\Omega)) &= k\left(\mathbf{I}_{p^{2}}+K_{pp}\right)(V\otimes V)\\ \operatorname{Var}(\log|\Omega|) &= \sum_{i=1}^{p}\psi_{1}\left(\dfrac{k-i+1}{2}\right) \end{align} $$ where $K_{pp}$ is a $p^{2}\times p^{2}$ commutation matrix such that $$ \begin{equation} K_{pp}\operatorname{vec}(C) = \operatorname{vec}(C') \end{equation} $$ for a $p\times p$ matrix $C$ and $\psi_{1}$ is the trigamma function. But I have no idea how to get the following covariance term. $$ \operatorname{Cov}(\operatorname{vec}(\Omega),\log|\Omega|) $$

Answer 1

$\newcommand{\D}{\textsf{D}}$ $\newcommand{\tr}{\text{tr}}$ $\newcommand{\vec}{\text{vec}}$ $\newcommand{\vech}{\text{vech}}$

I've derived it with the second order differential.

The log-likelihood is $$ \begin{align} \ell(\nu, \Sigma \mid W) & = C - \frac{\nu p}{2} \log 2 - \log\Gamma_p\left(\frac{\nu}{2}\right) - \frac{\nu}{2} \log |\Sigma| \\ & + \frac{\nu-p-1}{2} \log |W| -\frac{1}{2} \tr(\Sigma^{-1}W). \end{align} $$ Its differential at $(\nu_0, \Sigma_0)$ is $$ \begin{align} \D_{\nu_0,\Sigma_0}\ell & = 0 - \frac{p}{2} \log 2\,\D\nu - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu - \frac{1}{2}\log |\Sigma_0|\,\D\nu - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \\ & + \frac{1}{2}\log |W|\,\D\nu + \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr). \end{align} $$ Let's differentiate each term. $$ \D_{\nu_0,\Sigma_0}\left\{ - \frac{p}{2} \log 2\,\D\nu \right\} = 0. $$ $$ \D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu \right\} = - \frac{1}{4}\psi'_p\left(\frac{\nu_0}{2}\right)\D\nu\D\nu $$ $$ \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\log |\Sigma_0|\,\D\nu \right\} & = - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} $$ $$ \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \right\} & = \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = \frac{\nu_0}{2} {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma) - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} $$ $$ \D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2}\log |W|\,\D\nu \right\} = 0. $$ $$ \D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2} \tr(\Sigma_0^{-1}\D\Sigma\Sigma_0^{-1}W) \right\} = - 2 \times \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr). $$ Since $\mathbb{E}[W] = \nu_0\Sigma_0$, the expectation of the term above (which is the only term depending on $W$) is $$ - \nu_0\tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr) = - \nu_0 {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma). $$ Finally, note that $$ \vec(\D\Sigma) = D_p \vech(\D\Sigma), $$ where $D_p$ denotes the duplication matix, then the Fisher information matrix for the parameterization $\bigl(\nu, \text{vech}(\Sigma)\bigr)$ is $$ {\cal I}(\nu, \Sigma) = \begin{pmatrix} \frac{1}{4}\psi'_p\left(\frac{\nu}{2}\right) & \frac{1}{2}{\bigl(D_p' \vec(\Sigma^{-1})\bigr)}' \\ \frac{1}{2} D_p' \vec(\Sigma^{-1}) & \frac{\nu}{2} D_p'(\Sigma_0^{-1} \otimes \Sigma_0^{-1})D_p \end{pmatrix}. $$ Let's check:

psi1p <- function(p, a){ # second derivative of log Gamma_p
  sum(gsl::psi_1(a + (1-(1:p))/2))
}
p <- 2
Dp <- matrixcalc::duplication.matrix(p)
logL <- function(params, W){ # log likelihood
  Sigma <- matrix(Dp %*% params[-1], p, p)
  LaplacesDemon::dwishart(W, params[1], Sigma, log=TRUE)
}
# parameters 
nu <- 6
Sigma <- toeplitz(p:1)
# Information matrix obtained by simulations
nsims <- 10000
Wsims <- matrixsampling::rwishart(nsims, nu, Sigma)
Hsims <- array(NA_real_, dim=c(4,4,nsims))
for(i in 1:nsims){
  Hsims[,,i] <- numDeriv:: hessian(logL, 
                                   c(nu, Sigma[upper.tri(Sigma, diag=TRUE)]), 
                                   W=Wsims[,,i])
}
round(apply(-Hsims, 1:2, mean), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.32 -1.31  0.32
## [3,] -0.33 -1.31  3.28 -1.30
## [4,]  0.33  0.32 -1.30  1.31
# Exact information matrix
I11 <- psi1p(p, nu/2)/4
I21 <- t(Dp) %*% c(solve(Sigma)) / 2
I22 <- nu/2 * t(Dp) %*% (kronecker(solve(Sigma), solve(Sigma))) %*% Dp
round(rbind(c(I11, I21), cbind(I21, I22)), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.33 -1.33  0.33
## [3,] -0.33 -1.33  3.33 -1.33
## [4,]  0.33  0.33 -1.33  1.33