How to find the support of the posterior distribution to apply Metropolis-Hastings MCMC algorithm?

Question

I am trying to sample from a posterior distribution using a MCMC algorithm using the Metropolis-Hastings sampler.

How should I deal with the situations where I'm stuck in regions of the posterior with zero probability?

These regions are present because the posterior distribution is truncated and also due to numerical limitations on the computer the likelihood can become zero if you are very far from the mean. That is, say the likelihood is distributed normally; if you are 100 standard deviations away from the mean you get what appears as zero probability to the computer.

What I want to know is how to chose the initial value of the chain in order to be sure that it is contained in the support of the posterior.

Answer 1

This is an implementation problem since, theoretically, MCMC has no problem with truncated distributions.

Let $D$ be the support of your posterior. Just define your log-posterior as $-\infty$ if $\theta\not\in D$ and choose a suitable value on $D$ as an initial point.

For example, suppose that $x_1,\dots,x_n \stackrel{ind.}{\sim} \exp(\lambda)$ and that $\lambda\sim Unif(0,10)$. The following code shows how to implement a Metropolis-Hastings for this model using the package 'mcmc'.

library(mcmc)

set.seed(123)
x = rexp(100,1)  # The theoretical value of lambda is 1

# logposterior    
logpost = function(lambda){
if(lambda>0 & lambda <10) return(sum(dexp(x,lambda,log=T)))
else return(-Inf)
}

# Metropolis-Hastings

NS = 55000

out <- metrop(logpost, scale = .5, initial = 1, nbatch = NS)

out$accept

lambdap =   out$batch[ , 1][seq(5000,NS,25)] # posterior simulations after burning and thinning

hist(lambdap)