MCMC for infinite variance posteriors

Question

My question originated from Xi'an's suggestion to check integrability against the posterior in my nonlinear hierarchical model. I did not check it, but had possible infinity in mind and found out that one of my conditionals (which is inverse gamma distribution) have infinite variance. So that sampling resulted in chains often being far away at the tails.

So, this raised a question for me:

How to properly sample from distributions with infinite variances like inverse gamma distribution (with $ \alpha = 2 $) or Levy distribution or any other distribution with infinite variance? What does MCMC offer? I tried to search for papers dealing with such issues, but still no luck.

Answer 1

There is nothing wrong with infinite variance distributions, per se... For instance, simulating a Cauchy using rcauchy(10^3) produces a sample truly from a Cauchy distribution! Hence MCMC has no specific feature to "fight" for or against infinite variance distributions.

The difficulty with infinite variance distributions is at the Monte Carlo level, for instance if you want to compute $$ \mathfrak{I} = \int_0^\infty \sqrt{x} \dfrac{1}{\pi}\dfrac{1}{1+x^2} \,\text{d}x $$ the integral exists (and is finite), but using $$ \dfrac{1}{N} \sum_{i=1}^N \sqrt{|x_i|} $$ when the $x_i$'s are Cauchy leads to an infinite variance estimate. See, e.g.,

> expl=matrix(abs(rcauchy(10^6)),ncol=1000)
> est=apply(expl,2,mean)/2
> quantile(est,c(.9,.99,.999))
      90%       99%     99.9% 
   6.484375  37.393755 160.869406 

which shows that the estimator can get very large! And away from the true value

> integrate(function(x){sqrt(x)*dcauchy(x)},low=0,up=Inf)
0.7071078 with absolute error < 2e-05

In this case, you need to use importance sampling.