What is the deeper intuition behind the symmetric proposal distribution in the Metropolis-Hastings Algorithm?

Question

In the Metropolis-Hastings Algorithm, one usually considers a symmetric proposal distribution:

$$ J(\theta^*|\theta^{(s)}) $$

where $\theta^*$ is a proposal point and $\theta^{(s)}$ is the accepted value at iteration $s$. The symmetry states that:

$$ J(\theta^* = \theta_b|\theta^{(s)}= \theta_a) = J(\theta^* = \theta_a|\theta^{(s)}= \theta_b) $$

An example usually stated is a Normal or Uniform distribution. My questions are:

1) the Normal and Uniform are symmetric probability density functions themselves, is this notion of "symmetry" the same as the "symmetry" above?

2) Is there an intuitive way of seeing the deeper meaning behind the symmetry formula above? Why is it needed?

Thanks!

Answer 1

1) the Normal and Uniform are symmetric probability density functions themselves, is this notion of "symmetry" the same as the "symmetry" above?

Both distributions are symmetric around their mean. But the symmetry in Metropolis-Hastings is that $q(x|y)=q(y|x)$ which makes the ratio cancel in the Metropolis-Hastings acceptance probability. If one uses a Normal distribution not centered at the previous value in the Metropolis-Hastings proposal (as e.g. in the Langevin version), the Normal distribution remains symmetric as a distribution but the proposal distribution is no longer symmetric and hence it must appear in the Metropolis-Hastings acceptance probability.

2) Is there an intuitive way of seeing the deeper meaning behind the symmetry formula above? Why is it needed?

There is no particular depth in this special case, it simply makes life easier by avoiding the ratio of the proposals. It may save time or it may avoid computing complex or intractable densities. Note also that the symmetry depends on the parameterisation of the model: if one changes the parameterisation, a Jacobian appears and kills the symmetry.