4
$\begingroup$

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf

which claims that the mean acceptance probability when performing the Metropolis-Hastings algorithm with a standard normal target distribution and a normal proposal distribution is $$\frac{2}{\pi} \text{arctan}\left(\frac{2}{\sigma}\right)$$ where $\sigma$ is the standard deviation of the proposal. Can anyone help with a derivation?

$\endgroup$
3
$\begingroup$

The average acceptance probability is, under stationarity (i.e., $\theta\sim\mathcal N(0,1)$) \begin{align*}\require{amsmath} \mathbb P(\text{accept }\theta^\prime) &= \mathbb P(U\le e^{\theta^2/2-\theta'^2/2})&&\text{ratio of targets}\\ &=\mathbb E[\min\{1,e^{\theta^2/2-\theta'^2/2}\}]&&\text{mean probability}\\ &= \mathbb P(\theta'^2<\theta^2)&&\text{under the joint:}\\ &\qquad+\int_{|\theta'|>|\theta|} e^{\theta^2/2-\theta'^2/2} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}&&\text{target x proposal}\\ &= \int_{|\theta'|<|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}\\ &\qquad+\int_{|\theta'|>|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta'^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}\\ &= 2 \int_{|\theta'|<|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}&&\text{by symmetry}\\ &= 2 [\mathbb P\left(\theta'<|\theta| \right)-\mathbb P\left(\theta'<-|\theta| \right)]\\ &= 2 [\mathbb P\left(\theta'-\theta<|\theta|-\theta \right)-\mathbb P\left(\theta'-\theta<-|\theta|-\theta \right)]\\ &= 2 \mathbb E[\Phi(-2\theta/\sigma)\mathbb I_{\theta<0}-\Phi(-2\theta/\sigma)\mathbb I_{\theta>0}]&&\text{standardisation}\\ &= 2 \int_{-\infty}^0 \int_{2x/\sigma}^{-2x/\sigma} e^{-(x^2+y^2)/2}\frac{\text dx\text dy}{2\pi}\\ &=\int_{-\infty}^{\infty} \int_{2x/\sigma}^{-2x/\sigma} e^{-(x^2+y^2)/2}\frac{\text dx\text dy}{2\pi}&&\text{by symmetry}\\ &= \mathbb P(|Y/X|<2/\sigma)&&X,Y\text{ iid Normal}\\ &= \frac{2}{\pi}\arctan(2/\sigma) &&\text{Cauchy cdf} \end{align*} since $X/Y$ is distributed as a Cauchy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.