# Priors DURING Metropolis-Hastings-Random Walk chain (MCMC)

Suppose we are running a Metropolis-Hastings Random Walk chain (MHRW) targeting the unknown posterior distribution of a $$\theta$$, using data $$Y$$ and likelihood $$L$$. Since we do not know the posterior we will use instead the product of "likelihood $$\times$$ prior".

We will use the symbol $$p$$ for the prior distribution, and the symbol $$q$$ for the proposal density. We have just completed the $$j-1$$ step and have determined the corresponding element of the chain as $$\theta_{j-1}$$.

We now generate a proposal MHRW-style, using some zero-mean random variable $$v_j$$ as $$\theta^*_j = \theta_{j-1} + v_j$$

We then compute the acceptance probability (I abstract from the formal $$\min$$ formulation),

$$\alpha = \frac{L(Y \mid \theta^*_{j})\cdot p(\theta^*_{j}) }{L(Y \mid \theta_{j-1})\cdot p(\theta_{j-1})} \cdot \frac{q(\theta_{j-1}\mid\theta^*_{j})}{q(\theta^*_{j}\mid\theta_{j-1})}.$$

It is clear to me how we go about computing the two likelihoods $$L()$$ and the two proposal densities $$q()$$.

But what is that we do with the prior distributions?

What is the prior distribution of $$\theta^*_j$$?

What is the prior distribution of $$\theta_{j-1}$$?

Is it the "original", initial prior distribution that we have assumed for $$\theta$$ before activating the MHRW algorithm (and so common for both the current proposal and the previous accepted), or it should be the prior distribution of the MHRW itself which, (personification), standing at the end of completed period $$j-1$$ and getting ready to execute itself one more time, has amassed additional information on $$\theta$$ that we did not possess when we were standing at the beginning of period $$1$$?

Since this "additional information" has been acquired by using also the sample $$Y$$ through the previous iterations, it would appear by first principles, that we should stick with the initial-before-MHRW prior distribution.

But I thought I'd ask, following the well known advice "better silly than sorry".

What is the prior distribution of $$θ^∗_j$$? What is the prior distribution of $$θ_{j−1}$$? Is it the "original", initial prior distribution that we have assumed for $$\theta$$ before activating the MHRW algorithm?
That's right. It is the original prior for $$\theta$$ evaluated at the current value $$θ_{j−1}$$ and at the candidate value $$θ^∗_j$$.
Also, if the distribution of $$v_j$$ is symmetric about zero (in addition to having mean zero), then it holds that $$q(θ^∗_j|θ_{j−1}) = q(θ_{j−1}|θ^∗_j)$$ and the two terms cancel out. This is the original Metropolis algorithm.