Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf
which claims that the mean acceptance probability when performing the Metropolis-Hastings algorithm with a standard normal target distribution and a normal proposal distribution is $$\frac{2}{\pi} \text{arctan}\left(\frac{2}{\sigma}\right)$$ where $\sigma$ is the standard deviation of the proposal. Can anyone help with a derivation?
The average acceptance probability is, under stationarity (i.e., $\theta\sim\mathcal N(0,1)$) \begin{align*}\require{amsmath} \mathbb P(\text{accept }\theta^\prime) &= \mathbb P(U\le e^{\theta^2/2-\theta'^2/2})&&\text{ratio of targets}\\ &=\mathbb E[\min\{1,e^{\theta^2/2-\theta'^2/2}\}]&&\text{mean probability}\\ &= \mathbb P(\theta'^2<\theta^2)&&\text{under the joint:}\\ &\qquad+\int_{|\theta'|>|\theta|} e^{\theta^2/2-\theta'^2/2} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}&&\text{target x proposal}\\ &= \int_{|\theta'|<|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}\\ &\qquad+\int_{|\theta'|>|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta'^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}\\ &= 2 \int_{|\theta'|<|\theta|} e^{-(\theta-\theta')^2/2\sigma^2} e^{-\theta^2/2}\frac{\text d\theta\text d\theta'}{2\pi\sigma}&&\text{by symmetry}\\ &= 2 [\mathbb P\left(\theta'<|\theta| \right)-\mathbb P\left(\theta'<-|\theta| \right)]\\ &= 2 [\mathbb P\left(\theta'-\theta<|\theta|-\theta \right)-\mathbb P\left(\theta'-\theta<-|\theta|-\theta \right)]\\ &= 2 \mathbb E[\Phi(-2\theta/\sigma)\mathbb I_{\theta<0}-\Phi(-2\theta/\sigma)\mathbb I_{\theta>0}]&&\text{standardisation}\\ &= 2 \int_{-\infty}^0 \int_{2x/\sigma}^{-2x/\sigma} e^{-(x^2+y^2)/2}\frac{\text dx\text dy}{2\pi}\\ &=\int_{-\infty}^{\infty} \int_{2x/\sigma}^{-2x/\sigma} e^{-(x^2+y^2)/2}\frac{\text dx\text dy}{2\pi}&&\text{by symmetry}\\ &= \mathbb P(|Y/X|<2/\sigma)&&X,Y\text{ iid Normal}\\ &= \frac{2}{\pi}\arctan(2/\sigma) &&\text{Cauchy cdf} \end{align*} since $X/Y$ is distributed as a Cauchy.
