Let's say I have a model like: \begin{align} X\mid\mu,\Sigma_X &\sim \mathcal{N}(\mu,\Sigma_X)\\ \mu\mid m, \Sigma_\mu &\sim \mathcal{N}(m,\Sigma_\mu) \\ \Sigma_X\mid \Psi, c &\sim \mathcal{W}^{-1}(\Psi,c) \end{align} where $\Sigma_X$ is inverse-Wishart. This is, if I'm not mistaken, a classic Bayesian case with a Normal-inverse-Wishart conjugate prior. So the posterior predictive distribution is multivariate $t$-distributed.

Ultimately, I am interested in a simple analytic estimate of the covariance and/or entropy of the posterior predictive distribution of $X$. (This is because it is intended to be used in a loss function for optimization).

However, my situation should be a little simpler because $\Sigma_X$ is diagonal in my case. So using $\mathcal{W}^{-1}$ might not make sense.

I have estimates for $m$, $\Sigma_\mu$, as well as estimates of $\mathbb{E}[\Sigma_X]$ and $\text{Cov}(\Sigma_X)$, the latter being the covariances between the elements on the diagonal of $\Sigma_X$ (i.e. viewing $\Sigma_X$ as a vector of dimensionality $n$, we have $ \text{Cov}(\Sigma_X)\in\mathbb{R}^{n\times n} $). That should let me estimate $\Psi$ and $c$ too.

So here are my questions:

  • Since the covariance $\Sigma_X$ is diagonal, is there a more sensible prior for it? Essentially the inverse Wishart has been chosen for convenience. But I am not sure it makes sense for diagonal covariance matrices. Preferably, something analytically simple at the end (e.g. a normally distributed approximate posterior predictive distribution would be ideal, as occurs if I ignore all the information about the distribution $\Sigma_X$ and treat it as fixed - but this approach ignores the effect of the randomness of $\Sigma_X$, for which I'd like to account).

  • I am ok with throwing away some information in exchange for a simpler prior (all my estimates are going to be noisy anyway). One method that seems promising is the fact that the marginals of the diagonal elements of an inverse-Wishart distributed matrix are inverse gamma distributed. So I could throw away my info on the covariances of $\Sigma_X$ and treat each diagonal entry of $\Sigma_X$ as independent. I'm not sure how much that simplifies the posterior predictive distribution of $X$ though.

  • $\begingroup$ I am looking for a similar response. Have you found one ? Thank you in advance $\endgroup$
    – RobF
    Sep 22, 2019 at 10:19
  • $\begingroup$ If $\Sigma_X$ is diagonal, then the elements of $X$ are independent and hence you have many independent random variables instead of a random vector. As you mention, in this case the inverse gamma is the conjugate for the variances of the $X_i$ (the elements of $X$). I don't understand why you say you throw away info. I think it is the opposite: by avoiding the IW, you bring in the knowledge that $\Sigma_X$ is diagonal. $\endgroup$
    – papgeo
    Nov 14, 2019 at 13:27


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.