Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model

Question

How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution.

By Matrix-Normal distribution, I mean this distribution.

Answer 1

First, I'll assume by Matrix-Normal-Inverse-Wishart distribution, you mean $$ \begin{align} M &\sim \mathcal{MN}\left(M_0, \frac{1}{\lambda_1}U, \frac{1}{\lambda_2}V\right)\\ U &\sim \mathcal{W}^{-1}\left(\Psi,\nu\right)\\ V &\sim \mathcal{W}^{-1}\left(\Phi,\eta\right)\\ \end{align} $$ To prove that a prior is conjugate, you check that the posterior has the same form as the prior: $$ P(M,U,V|X) =P(X|M, \lambda_1, \lambda_2, U, V)P(U|\Psi, \nu)P(V|\Phi,\eta) $$ Typically this is done in log space. I'll skip writing out all the expressions, but it turns out that (via completing the matrix quadratic form) there is an extra term $$-\frac{1}{2}(1+\lambda_1\lambda_2)\text{tr}\left[V^{-1}\left(\frac{\sum_k X_k+\lambda_1\lambda_2M_0}{k + \lambda_1\lambda_2}\right)^{T}U^{-1}\left(\frac{\sum_k X_k+\lambda_1\lambda_2M_0}{k + \lambda_1\lambda_2}\right)\right]$$ which means that this formulation of the Matrix-Normal-Inverse-Wishart distribution is not conjugate. However, other formulations may be conjugate. For example, if we instead fix one of $U$ or $V$ as a hyperparameter, then due to the circular property of the trace, the form of the posterior will match the prior. Alternatively, if $n=p$, we could state that $V$ depends on $U$: $$ \begin{align} M &\sim \mathcal{MN}\left(M_0, \frac{1}{\lambda_1}U, \frac{1}{\lambda_2}V\right)\\ V &\sim \mathcal{W}^{-1}\left(Z^TU^{-1}Z,\eta\right)\\ U &\sim \mathcal{W}^{-1}\left(\Psi,\nu\right)\\ \end{align} $$ for some hyperparameter $Z\in \mathbb{R}^{n \times n}$. Remember, there is nothing special about a conjugate prior except for providing closed-form posteriors. For example, if I want to make inferences about both $U$ and $V$ simultaneously, then the conjugate prior made by fixing one is a poor choice.