Is Inverse-Wishart a conjugate prior for Wishart likelihood?

Question

Suppose I have a noisy observation $Z$ of a covariance matrix $F$, given a prior on $F: p(F)$, I would like to find the posterior of $p(F|Z)$, does the following specification forms conjugacy?:

$$ F \sim IW(v, F_0)$$ $$Z|F \sim W(v_k , \frac{F}{v_k})$$ $IW$ denotes Inverse-Wishart, $W$ denotes Wishart. $Z$ is a noisy observation of $F$ since $E[Z|F] = v_k \frac{F}{v_k} = F$.

My derivation shows that they are conjugate but I cannot find anything similar online, so my derivations should be incorrect. Where went wrong?:

$$p(F|Z) \propto p(F) p(Z|F) $$ $$\propto |F|^{-\frac{v + p +1}{2}}e^{-\frac{1}{2} Tr(F_0 F^{-1})} |F|^{-\frac{d}{2}}e^{-\frac{1}{2} Tr(v_k F^{-1} Z)}$$ $$=|F|^{-\frac{v + p +1}{2}}e^{-\frac{1}{2} Tr(F_0 F^{-1})} |F|^{-\frac{v_k}{2}}e^{-\frac{1}{2} Tr(v_k Z F^{-1})} \text{ (by $Tr(AB) = Tr(BA)$)}$$ $$=|F|^{-\frac{v+p+1+v_k}{2}} e^{-\frac{1}{2}Tr((F_0 + v_k Z)F^{-1})}$$ $$\sim IW(v+v_k, F_0 + v_k Z)$$

Does the Bayes' Theorem apply to two random matrices?

Answer 1

This is actually quite a well-known result in Bayesian statistics (see e.g., Evans 1965, Chen 1979, Dickey, Lindley and Press 1985 and Leonard and Hsu 1992). In most of the literature on Bayesian analysis of this problem it is typical for the analysis to be framed in terms of the precision matrix $\mathbf{P} = \mathbf{F}^{-1}$ rather than the variance matrix $\mathbf{F}$, so the usual way your result would be framed is entirely in terms of the Wishart distribution:

$$\begin{align} \mathbf{P} &\sim \text{Wishart}(\text{df} = v, \text{Precision} = \mathbf{P}_0), \\[12pt] \mathbf{Z}|\mathbf{P} &\sim \text{Wishart}(\text{df} = v_k, \text{Precision} = v_k \mathbf{P}). \\[6pt] \end{align}$$

This is equivalent to your formulation:

$$\begin{align} \mathbf{F} &\sim \text{InvWishart}(\text{df} = v, \text{Variance} = \mathbf{F}_0), \\[6pt] \mathbf{Z}|\mathbf{F} &\sim \text{Wishart} \bigg( \text{df} = v_k, \text{Variance} = \frac{\mathbf{F}}{v_k} \bigg). \\[6pt] \end{align}$$

You appear to have made some minor errors in your algebra, but those do not affect your ultimate result, which is correct. From the Wishart distribution you have likelihood function:

$$L_\mathbf{z}(\mathbf{F}) \propto |\mathbf{F}|^{-v_k/2} \exp \bigg( -\frac{1}{2} \text{tr}(v_k \mathbf{F}^{-1} \mathbf{z}) \bigg).$$

Taking the prior density $\pi$ from the inverse-Wishart distribution gives the form:

$$\pi(\mathbf{F}) \propto |\mathbf{F}|^{-(v+p+1)/2} \exp \bigg( -\frac{1}{2} \cdot \text{tr}(\mathbf{F}_0 \mathbf{F}^{-1}) \bigg),$$

which gives the posterior form:

$$\begin{align} \pi(\mathbf{F} | \mathbf{z}) &\propto |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \cdot \text{tr}(\mathbf{F}_0 \mathbf{F}^{-1}) -\frac{1}{2} \text{tr}(v_k \mathbf{F}^{-1} \mathbf{z})\bigg) \\[6pt] &= |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \bigg[ \text{tr}(\mathbf{F}_0 \mathbf{F}^{-1}) + \text{tr}(v_k \mathbf{F}^{-1} \mathbf{z}) + \bigg] \bigg) \\[6pt] &= |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \bigg[ \text{tr}(\mathbf{F}_0 \mathbf{F}^{-1}) + \text{tr}(v_k \mathbf{z} \mathbf{F}^{-1}) \bigg] \bigg) \\[6pt] &= |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \bigg[ \text{tr}(\mathbf{F}_0 \mathbf{F}^{-1} + v_k \mathbf{z} \mathbf{F}^{-1}) \bigg] \bigg) \\[6pt] &= |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \bigg[ \text{tr}((\mathbf{F}_0 + v_k \mathbf{z}) \mathbf{F}^{-1}) \bigg] \bigg) \\[6pt] &= |\mathbf{F}|^{-(v+v_k+p+1)/2} \exp \bigg( -\frac{1}{2} \bigg[ \text{tr}( (\mathbf{F}_0 + v_k \mathbf{z}) \mathbf{F}^{-1}) \bigg] \bigg) \\[12pt] &\propto \text{InvWishart}(\mathbf{F} | v+v_k, \mathbf{F}_0 + v_k \mathbf{z}). \\[14pt] \end{align}$$