3
$\begingroup$

I have been struggling computing the Fisher's information of the Wishart distribution. I'll write what I have gone through. Let's $\Omega$ denote a $p\times p$ Wishart random variate denoted by $\mathcal{W}(k,V)$ where $k$ is the degrees of freedom and $V$ a positive definite scale matrix. If we write $\mathcal{W}(\Omega\,|\,k,V)$ for the density function, $$ \begin{align} \nabla_{\operatorname{vech}(V)}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}D_{p}'\left(V^{-1}\otimes V^{-1}\right)D_{p}\operatorname{vech}(\Omega)-\dfrac{k}{2}D_{p}\operatorname{vech}\left(V^{-1}\right)\\ \nabla_{k}\log\mathcal{W}(\Omega\,|\,k,V) &= \dfrac{1}{2}\log|\Omega|-\dfrac{1}{2}\log|V|-\dfrac{p}{2}\log 2-\dfrac{1}{2}\sum_{i=1}^{p}\psi\left(\dfrac{k+1-i}{2}\right) \end{align} $$ where $D_{p}$ is the unique duplication matrix such that $D_{p}\operatorname{vech}(A)=\operatorname{vec}(A)$, $\otimes$ Kronecker product, and $\psi$ digamma function.

Is there a closed-form expression for the Fisher information matrix of the Wishart distribution?


UPDATE I have computed the following two terms: $$ \begin{align}\operatorname{Var}(\operatorname{vec}(\Omega)) &= k\left(\mathbf{I}_{p^{2}}+K_{pp}\right)(V\otimes V)\\ \operatorname{Var}(\log|\Omega|) &= \sum_{i=1}^{p}\psi_{1}\left(\dfrac{k-i+1}{2}\right) \end{align} $$ where $K_{pp}$ is a $p^{2}\times p^{2}$ commutation matrix such that $$ \begin{equation} K_{pp}\operatorname{vec}(C) = \operatorname{vec}(C') \end{equation} $$ for a $p\times p$ matrix $C$ and $\psi_{1}$ is the trigamma function. But I have no idea how to get the following covariance term. $$ \operatorname{Cov}(\operatorname{vec}(\Omega),\log|\Omega|) $$

$\endgroup$
2
$\begingroup$

$\newcommand{\D}{\textsf{D}}$ $\newcommand{\tr}{\text{tr}}$ $\newcommand{\vec}{\text{vec}}$

I've derived it with the second order differential.

The log-likelihood is $$ \begin{align} \ell(\nu, \Sigma \mid W) & = C - \frac{\nu p}{2} \log 2 - \log\Gamma_p\left(\frac{\nu}{2}\right) - \frac{\nu}{2} \log |\Sigma| \\ & + \frac{\nu-p-1}{2} \log |W| -\frac{1}{2} \tr(\Sigma^{-1}W). \end{align} $$ Its differential at $(\nu_0, \Sigma_0)$ is $$ \begin{align} \D_{\nu_0,\Sigma_0}\ell & = 0 - \frac{p}{2} \log 2\,\D\nu - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu - \frac{1}{2}\log |\Sigma_0|\,\D\nu - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \\ & + \frac{1}{2}\log |W|\,\D\nu + \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr). \end{align} $$ Let's differentiate each term. $$ \D_{\nu_0,\Sigma_0}\left\{ - \frac{p}{2} \log 2\,\D\nu \right\} = 0. $$ $$ \D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\psi_p\left(\frac{\nu_0}{2}\right)\D\nu \right\} = - \frac{1}{4}\psi'_p\left(\frac{\nu_0}{2}\right)\D\nu\D\nu $$ $$ \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{1}{2}\log |\Sigma_0|\,\D\nu \right\} & = - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} $$ $$ \begin{align} \D_{\nu_0,\Sigma_0}\left\{ - \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr) \right\} & = \frac{\nu_0}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu - \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\bigr)\D\nu \\ & = \frac{\nu_0}{2} {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma) - \frac{1}{2} {\vec(\D\Sigma)}'\vec(\Sigma_0^{-1})\D\nu. \end{align} $$ $$ \D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2}\log |W|\,\D\nu \right\} = 0. $$ $$ \D_{\nu_0,\Sigma_0}\left\{ \frac{1}{2} \tr(\Sigma_0^{-1}\D\Sigma\Sigma_0^{-1}W) \right\} = - 2 \times \frac{1}{2} \tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}W\bigr). $$ Since $\mathbb{E}[W] = \nu_0\Sigma_0$, the expectation of the term above (which is the only term depending on $W$) is $$ - \nu_0\tr\bigl(\Sigma_0^{-1}(\D\Sigma)\Sigma_0^{-1}(\D\Sigma)\bigr) = - \nu_0 {\vec(\D\Sigma)}' (\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \vec(\D\Sigma). $$ Finally, the Fisher information matrix for the parameterization $\bigl(\nu, \text{vech}(\Sigma)\bigr)$ is $$ {\cal I}(\nu, \Sigma) = \begin{pmatrix} \frac{1}{4}\psi'_p\left(\frac{\nu}{2}\right) & \frac{1}{2}{\bigl(D_p' \vec(\Sigma^{-1})\bigr)}' \\ \frac{1}{2} D_p' \vec(\Sigma^{-1}) & \frac{\nu}{2}(\Sigma_0^{-1} \otimes \Sigma_0^{-1}) \end{pmatrix}. $$ Let's check:

psi1p <- function(p, a){ # second derivative of log Gamma_p
  sum(gsl::psi_1(a + (1-(1:p))/2))
}
p <- 2
Dp <- matrixcalc::duplication.matrix(p)
logL <- function(params, W){ # log likelihood
  Sigma <- matrix(Dp %*% params[-1], p, p)
  LaplacesDemon::dwishart(W, params[1], Sigma, log=TRUE)
}
# parameters 
nu <- 6
Sigma <- toeplitz(p:1)
# Information matrix obtained by simulations
nsims <- 10000
Wsims <- matrixsampling::rwishart(nsims, nu, Sigma)
Hsims <- array(NA_real_, dim=c(4,4,nsims))
for(i in 1:nsims){
  Hsims[,,i] <- numDeriv:: hessian(logL, 
                                   c(nu, Sigma[upper.tri(Sigma, diag=TRUE)]), 
                                   W=Wsims[,,i])
}
round(apply(-Hsims, 1:2, mean), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.32 -1.31  0.32
## [3,] -0.33 -1.31  3.28 -1.30
## [4,]  0.33  0.32 -1.30  1.31
# Exact information matrix
I11 <- psi1p(p, nu/2)/4
I21 <- t(Dp) %*% c(solve(Sigma)) / 2
I22 <- nu/2 * t(Dp) %*% (kronecker(solve(Sigma), solve(Sigma))) %*% Dp
round(rbind(c(I11, I21), cbind(I21, I22)), 2)
##       [,1]  [,2]  [,3]  [,4]
## [1,]  0.22  0.33 -0.33  0.33
## [2,]  0.33  1.33 -1.33  0.33
## [3,] -0.33 -1.33  3.33 -1.33
## [4,]  0.33  0.33 -1.33  1.33
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.