# How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC:

• target distribution: $$\pi(x)$$
• proposal distribution: $$p(y|x)$$
• acceptance: $$\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$$

Here are some examples of a proposal distribution:

• $$p(y|x)$$ density of $$U(x-\sigma, x+\sigma)$$
• $$p(y|x)$$ density of $$N(x,\sigma^2).$$

Then how to understand the statements that:

1. large $$\alpha$$ or $$\sigma^2$$ will lower the acceptance;

2. small $$\alpha$$ or $$\sigma^2$$ will increase the correlation.

I cannot mathematically deduce the result from the acceptance formula.

• The intuition is that if your chain at iteration $i$ is $x_{i}$ and you propose a candidate that it is near $x^{'}_{cand}=x_{i}+0.1$ ($a$ small and $\sigma^{2}$ small) it is more likely to be accepted than a candidate $x_{cand}^{'}=x_{i}+1000$ ($a$ big and $\sigma^{2}$ big). So, obviously in the first case if you accept your candidate you will have higher correlation because it is almost the same with your $x_{i}$ Jul 19, 2021 at 12:21
• @Fiodor1234 agree on the intuition, actually I want to see the mathematical proof, especially for the first one. Jul 19, 2021 at 12:22
• Sure, I think I can demonstrate one Jul 19, 2021 at 12:25

Assuming a target distribution with unrestricted support $$\mathfrak X$$, consider the simple random walk proposal written as $$y=x+\sigma\epsilon\,,$$ where $$\epsilon$$ is a unit-variance symmetric white noise. Then $$\dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}=\dfrac{\pi(y)}{\pi(x)}=\dfrac{\pi(x+\sigma\epsilon)}{\pi(x)}$$ which a.s. converges to $$1$$ when $$\sigma$$ goes to zero and to $$0$$ when $$\sigma$$ goes to infinity.
• Sorry I cannot convince by above deduction, you should proof $P\Big(U\leq \min(1,\dfrac{\pi(x+\sigma\epsilon)}{\pi(x)})\Big)$ is an decreasing function of $\sigma.$ Jul 19, 2021 at 12:41
• "to 0 when $\sigma$ goes to infinity" why? Jul 19, 2021 at 12:44
• The property does not hold for all $\sigma$'s but only for small enough and large enough $\sigma$'s. Jul 19, 2021 at 12:54
• even for the large enough $\sigma,$ why goes to 0? Or everything here is "almost surely"? Jul 19, 2021 at 13:06