Metropolis-Hastings rejects a lot I'm implementing an algorithm to make inference about some parameters.
Let's say I would like to make inference about a parameter set $\theta$ using Gibbs sampling and then, using the updated $\theta$, I would like to make inference about a parameter vector $V$ using random walk Metropolis Hastings.
I know my first "few" samples $\hat{\theta}_1, ..., \hat{\theta}_k$ should be omitted because of the "burn-in" period. These parameters are sampled far away from the true distribution of these parameters.
But the problem I enter is the following: I need for instance $\hat{\theta}_1$ to make inference about $V_1$. But since ${\theta}_1$ is a very poor estimate, my Metropolis-Hastings algorithm rejects all time. The numerator and denominator of my "accept-rule" is 0. How can I solve this problem?
Any help is appreciated.
 A: The acceptance/rejection step in the MH algorithm is based on the ratio of the density values of the previous parameter value and the newly proposed value.  It shouldn't matter if the previous value is a poor estimator --- all that matters is the relative density at the previous parameter value and the proposed parameter value.  To see this, suppose we have generated $m$ iterations of the algorithm, and we are now generating iteration $m+1$, updating the $i$th element of $\boldsymbol{\theta}$.  Then the newly proposed parameter vector is:
$$\hat{\boldsymbol{\theta}}_{m+1,i} \equiv (\hat{\theta}_1^{(m+1)}, ..., \hat{\theta}_{i}^{(m+1)}, \hat{\theta}_{i+1}^{(m)}, ..., \hat{\theta}_k^{(m)}),$$
and the corresponding acceptance probability cut-off (allowing values greater than one) is:
$$A(\hat{\boldsymbol{\theta}}_{m+1,i},\hat{\boldsymbol{\theta}}_{m+1,i-1}) 
\equiv \frac{p(\hat{\boldsymbol{\theta}}_{m+1,i})}{p(\hat{\boldsymbol{\theta}}_{m+1,i-1})} \cdot \frac{g(\hat{\theta}_i^{(m+1)}|\hat{\theta}_i^{(m)})}{g(\hat{\theta}_{i}^{(m)}|\hat{\theta}_i^{(m+1)})}.$$
In the case where the proposal distribution $g$ is symmetric around the conditioning value, the latter term disappears, so we then have:
$$A(\hat{\boldsymbol{\theta}}_{m+1,i},\hat{\boldsymbol{\theta}}_{m+1,i-1}) 
= \frac{p(\hat{\boldsymbol{\theta}}_{m+1,i})}{p(\hat{\boldsymbol{\theta}}_{m+1,i-1})}.$$
Now, if the previous parameter $\hat{\boldsymbol{\theta}}_{m+1,i-1}$ is a particularly poor estimator, this just means that the denominator density is small.  So long as the newly proposed value has a higher density, it will automatically be accepted.  If it has a lower density, then it may be rejected, but this will only occur with high probability if it has a much lower density than the present point.  The only time you should have genuine troubles where your algorithm is rejecting lots of values is when you are stuck in an area of high probability that is an "island" surrounded by areas of low probability, and your proposed values have a high enough bandwidth that they tend to land in the lower-probability areas around the "island".

Arithmetic underflow: You say in your question that the numerator and denominator for your computation are both zero, which suggests that the problem here is really an issue of arithmetic underflow (i.e., the computer is rounding non-zero density values in the numerator and denominator to zero).  You can deal with this by working in log-space, using the log-probability of acceptance:
$$\ln A(\hat{\boldsymbol{\theta}}_{m+1,i},\hat{\boldsymbol{\theta}}_{m+1,i-1}) 
= \ln p(\hat{\boldsymbol{\theta}}_{m+1,i}) - \ln p(\hat{\boldsymbol{\theta}}_{m+1,i-1}).$$
It should be reasonably simple to compute this log-probability using the log-densities of whatever distribution you are working with.  (For example, when you use distribution functions in R there is an option to set log = TRUE to get the log-density instead of the density.)  You can then exponentiate this value to obtain the probability cut-off of interest.  This should avoid the arithmetic underflow problem, and give you non-zero acceptance probabilities in at least one direction.  If you still have troubles with your algorithm, try altering the bandwidth of the proposal distribution.
