7
$\begingroup$

During Gibbs sampling, I need to sample a multivariate Gaussian distribution. But rmvnorm return errors sometimes because the covariance matrix is not definite and I found the determinant of it is, say, $-10^{-20}$. I suspect this is caused by the numerical errors. What should I do to handle this?

$\endgroup$
2
  • $\begingroup$ Why the covariance matrix is not definite? How did the errors occur? Could you tell us more? $\endgroup$
    – Tim
    Apr 2, 2017 at 8:53
  • $\begingroup$ @Tim it can happen eg because there are some terms of the Gaussian with negligible coefficients. You could reduce the dimensionality of the covariance matrix to compensate, but in general that is much more complicated than the very much simpler and effective solution I have detailed in my answer below. $\endgroup$ Apr 2, 2017 at 12:47

1 Answer 1

5
$\begingroup$

You can add a tiny epsilon to the diagonal, say $1e-8$ or similar, to the covariance matrix.

So, let's say your covariance matrix is $\mathbf{\Sigma}$. And when you do stuff that involves $\text{inv}(\mathbf{\Sigma})$, or similar, it doesn't work very well, because your $\Sigma$ is not positive definite. So, therefore you can update your $\mathbf{\Sigma}$ as follows:

$$ \mathbf{\Sigma}' = \mathbf{\Sigma} + 10^{-8} \mathbf{I} $$

Now your new $\mathbf{\Sigma}'$ matrix is positive definite, and your numerical calculations will work much more smoothly :-)

I've used this technique in the past, and it works very well for me.

$\endgroup$
3
  • 1
    $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? You can also turn it into a comment. $\endgroup$ Apr 2, 2017 at 12:18
  • $\begingroup$ Fleshed out the answer to explain how to update $\Sigma$ with the 1e-8, and the statement that this makes the matrix become positive definite. $\endgroup$ Apr 2, 2017 at 12:44
  • 1
    $\begingroup$ @gung Thank you for your clear suggestion for how to improve my reply. Much appreciated :-) $\endgroup$ Apr 2, 2017 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.