2
$\begingroup$

We know the Metropolis Hastings (MH) in MCMC:

  • target distribution: $\pi(x)$
  • proposal distribution: $p(y|x)$
  • acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$

Here are some examples of a proposal distribution:

  • $p(y|x)$ density of $U(x-\sigma, x+\sigma)$
  • $p(y|x)$ density of $N(x,\sigma^2).$

Then how to understand the statements that:

  1. large $\alpha$ or $\sigma^2$ will lower the acceptance;

  2. small $\alpha$ or $\sigma^2$ will increase the correlation.

I cannot mathematically deduce the result from the acceptance formula.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ The intuition is that if your chain at iteration $i$ is $x_{i}$ and you propose a candidate that it is near $x^{'}_{cand}=x_{i}+0.1$ ($a$ small and $\sigma^{2}$ small) it is more likely to be accepted than a candidate $x_{cand}^{'}=x_{i}+1000$ ($a$ big and $\sigma^{2}$ big). So, obviously in the first case if you accept your candidate you will have higher correlation because it is almost the same with your $x_{i}$ $\endgroup$
    – Fiodor1234
    Jul 19, 2021 at 12:21
  • $\begingroup$ @Fiodor1234 agree on the intuition, actually I want to see the mathematical proof, especially for the first one. $\endgroup$
    – user6703592
    Jul 19, 2021 at 12:22
  • $\begingroup$ Sure, I think I can demonstrate one $\endgroup$
    – Fiodor1234
    Jul 19, 2021 at 12:25

1 Answer 1

Reset to default
3
$\begingroup$

Assuming a target distribution with unrestricted support $\mathfrak X$, consider the simple random walk proposal written as $$y=x+\sigma\epsilon\,,$$ where $\epsilon$ is a unit-variance symmetric white noise. Then $$\dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}=\dfrac{\pi(y)}{\pi(x)}=\dfrac{\pi(x+\sigma\epsilon)}{\pi(x)}$$ which a.s. converges to $1$ when $\sigma$ goes to zero and to $0$ when $\sigma$ goes to infinity.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Sorry I cannot convince by above deduction, you should proof $P\Big(U\leq \min(1,\dfrac{\pi(x+\sigma\epsilon)}{\pi(x)})\Big)$ is an decreasing function of $\sigma.$ $\endgroup$
    – user6703592
    Jul 19, 2021 at 12:41
  • $\begingroup$ "to 0 when $\sigma$ goes to infinity" why? $\endgroup$
    – user6703592
    Jul 19, 2021 at 12:44
  • $\begingroup$ The property does not hold for all $\sigma$'s but only for small enough and large enough $\sigma$'s. $\endgroup$
    – Xi'an
    Jul 19, 2021 at 12:54
  • $\begingroup$ even for the large enough $\sigma,$ why goes to 0? Or everything here is "almost surely"? $\endgroup$
    – user6703592
    Jul 19, 2021 at 13:06
  • $\begingroup$ That the density goes to zero at infinity is a necessary condition for the density to be integrable. $\endgroup$
    – Xi'an
    Jul 19, 2021 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.