# Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very convenient but since its form is $p(\Sigma) \propto |\Sigma|^{-k}$ I'm not sure if it prevents them or not, because it is this term (which equals $\sigma^{-1}$ in the univariate case) that causes the problem in the first place according to the answer mentioned above. Would only proper priors do the trick?

• For covariance matrices you need a prior with support over positive-definite matrices. The Inverse-Wishart distribution is that prior. – kedarps May 17 '18 at 14:59
• @kedarps I think you should make that into an answer, because it is :) – conjectures May 18 '18 at 12:27
• The Stan Prior Choice Recommendation wiki (github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations) lists several options. I'm not an expert but I would recommend splitting the covariance matrix into a correlation $C$ matrix and a diagonal matrix of standard deviations $S$; then $SCS$ is your covariance matrix and you can specify priors separately for each. I hear the LKJ prior is useful for correlation matrices. I hope this helps! – Maurits M May 18 '18 at 13:50

## 1 Answer

The singularity issue that is mentioned in the answer is a problem with the MLE estimation for the GMM, when estimated using the expectation-maximization (EM) algorithm. If you use a Bayesian approach, you will not face singularity issues.

Typically, for covariance matrices you need a prior with support over positive-definite matrices which can be achieved by using the Inverse-Wishart prior.