# Appropriate Distribution for Diagonal Covariance Matrices

Let's say I have a model like: \begin{align} X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\ \mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\ \Sigma_X\mid \Psi, c &\sim \mathcal{W}^{-1}(\Psi,c) \end{align} where $$\Sigma_X$$ is inverse-Wishart. This is, if I'm not mistaken, a classic Bayesian case with a Normal-inverse-Wishart conjugate prior. So the posterior predictive distribution is multivariate $$t$$-distributed.

Ultimately, I am interested in a simple analytic estimate of the covariance and/or entropy of the posterior predictive distribution of $$X$$. (This is because it is intended to be used in a loss function for optimization).

However, my situation should be a little simpler because $$\Sigma_X$$ is diagonal in my case. So using $$\mathcal{W}^{-1}$$ might not make sense.

I have estimates for $$m$$, $$\Sigma_\mu$$, as well as estimates of $$\mathbb{E}[\Sigma_X]$$ and $$\text{Cov}(\Sigma_X)$$, the latter being the covariances between the elements on the diagonal of $$\Sigma_X$$ (i.e. viewing $$\Sigma_X$$ as a vector of dimensionality $$n$$, we have $$\text{Cov}(\Sigma_X)\in\mathbb{R}^{n\times n}$$). That should let me estimate $$\Psi$$ and $$c$$ too.

So here are my questions:

• Since the covariance $$\Sigma_X$$ is diagonal, is there a more sensible prior for it? Essentially the inverse Wishart has been chosen for convenience. But I am not sure it makes sense for diagonal covariance matrices. Preferably, something analytically simple at the end (e.g. a normally distributed approximate posterior predictive distribution would be ideal, as occurs if I ignore all the information about the distribution $$\Sigma_X$$ and treat it as fixed - but this approach ignores the effect of the randomness of $$\Sigma_X$$, for which I'd like to account).

• I am ok with throwing away some information in exchange for a simpler prior (all my estimates are going to be noisy anyway). One method that seems promising is the fact that the marginals of the diagonal elements of an inverse-Wishart distributed matrix are inverse gamma distributed. So I could throw away my info on the covariances of $$\Sigma_X$$ and treat each diagonal entry of $$\Sigma_X$$ as independent. I'm not sure how much that simplifies the posterior predictive distribution of $$X$$ though.

• I am looking for a similar response. Have you found one ? Thank you in advance – RobF Sep 22 '19 at 10:19
• If $\Sigma_X$ is diagonal, then the elements of $X$ are independent and hence you have many independent random variables instead of a random vector. As you mention, in this case the inverse gamma is the conjugate for the variances of the $X_i$ (the elements of $X$). I don't understand why you say you throw away info. I think it is the opposite: by avoiding the IW, you bring in the knowledge that $\Sigma_X$ is diagonal. – papgeo Nov 14 '19 at 13:27