Proving Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model How does one prove that the Matrix-Normal-Inverse-Wishart distribution is a conjugate prior for a Linear Model? This prior is a generalization of the Normal-Inverse-Wishart Distribution.
By Matrix-Normal distribution, I mean this distribution.
 A: First, I'll assume by Matrix-Normal-Inverse-Wishart distribution, you mean
$$
\begin{align}
M &\sim  \mathcal{MN}\left(M_0, \frac{1}{\lambda_1}U, \frac{1}{\lambda_2}V\right)\\
U &\sim \mathcal{W}^{-1}\left(\Psi,\nu\right)\\
V &\sim \mathcal{W}^{-1}\left(\Phi,\eta\right)\\
\end{align}
$$
To prove that a prior is conjugate, you check that the posterior has the same form as the prior:
$$
P(M,U,V|X) =P(X|M, \lambda_1, \lambda_2, U, V)P(U|\Psi, \nu)P(V|\Phi,\eta)
$$
Typically this is done in log space. I'll skip writing out all the expressions, but it turns out that (via completing the matrix quadratic form) there is an extra term $$-\frac{1}{2}(1+\lambda_1\lambda_2)\text{tr}\left[V^{-1}\left(\frac{\sum_k X_k+\lambda_1\lambda_2M_0}{k + \lambda_1\lambda_2}\right)^{T}U^{-1}\left(\frac{\sum_k X_k+\lambda_1\lambda_2M_0}{k + \lambda_1\lambda_2}\right)\right]$$
which means that this formulation of the Matrix-Normal-Inverse-Wishart distribution is not conjugate. However, other formulations may be conjugate. For example, if we instead fix one of $U$ or $V$ as a hyperparameter, then due to the circular property of the trace, the form of the posterior will match the prior. Alternatively, if $n=p$, we could state that $V$ depends on $U$:
$$
\begin{align}
M &\sim  \mathcal{MN}\left(M_0, \frac{1}{\lambda_1}U, \frac{1}{\lambda_2}V\right)\\
V &\sim \mathcal{W}^{-1}\left(Z^TU^{-1}Z,\eta\right)\\
U &\sim \mathcal{W}^{-1}\left(\Psi,\nu\right)\\
\end{align}
$$
for some hyperparameter $Z\in \mathbb{R}^{n \times n}$. Remember, there is nothing special about a conjugate prior except for providing closed-form posteriors. For example, if I want to make inferences about both $U$ and $V$ simultaneously, then the conjugate prior made by fixing one is a poor choice.
