2
$\begingroup$

I've been reading about rejection sampling for Bayesian Statistics, and I have the following example:

Given $y\sim N(\theta,1)$ , $p(\theta)\sim Cauchy(0,1)$, $q(\theta |y)=p(\theta |y)p(\theta )$, $g(\theta )\sim N(y,1)$ and y=1

Then, the example says: $\frac { q(\theta |y) }{ g(\theta ) } =\frac { exp\left\{ -\frac { { (y-\theta ) }^{ 2 } }{ 2 } \right\} \cdot \frac { 1 }{ 1+{ \theta }^{ 2 } } }{ \frac { 1 }{ \sqrt { 2\pi } } exp\left\{ -\frac { { (\theta -y) }^{ 2 } }{ 2 } \right\} } =\frac { \sqrt { 2\pi } }{ 1+{ \theta }^{ 2 } } \le \sqrt { 2\pi }$ $ \\ $

I understand almost everything, but the last step is not so clear for me (the inequality), because according that, $M=\sqrt { 2\pi } \\$

But I read the following definition: $M=\underset { \theta }{ sup } \quad \frac { q(\theta |y) }{ g(\theta ) } $, and I think $M=\frac{\sqrt { 2\pi } }{2}$

My second doubt is about the implementation Of this example with R. Assuming the first M, I use the following code

th<-rep(NA,1000)
i<-0
while(i<1000){
theta1<-rnorm(1,1,1)
u<-runif(1)
if(u<=1/(1+(theta1^2))){
i<-i+1
th[i]<-theta1
}
}
th
hist(th)

Am I implementing it in the correct way? Is it correct my histogram of the posterior? I

$\endgroup$
1
$\begingroup$

The first question is unclear: The inequality$$\dfrac{q(y|\theta)}{g(\theta)}=1\big/\frac{1}{\sqrt{2\pi}}{\{1+\theta^2\}}=\dfrac{\sqrt{2\pi}}{1+\theta^2}\le \sqrt{2\pi}$$ is correct when using $q(y|\theta)=\exp\{-(\theta-y)^2/2\}\big/\{1+\theta^2\}$ instead of the properly normalised density (which normalisation constant is not known in closed form). How would you derive that the upper bound is $\sqrt{2\pi}/2$ instead?

Checking your algorithm produces the right answer is straightforward: add the target density on top of your histogram:

enter image description here

which is produced by the following R code

pos<-function(x){dnorm(x,1,1)*dcauchy(x)}
normaz=integrate(pos,-50,150)$val
hist(th,prob=TRUE,nclass=57,col="wheat")
curve(pos(x)/normaz,add=TRUE,lwd=2,col="sienna3")
$\endgroup$
  • $\begingroup$ You're right... I was evaluating $\frac{\sqrt(2\pi)}{1+ {\theta}^{2}}$ at the MLE (1 in this case). Thanks a lot! $\endgroup$ – Red Noise Feb 25 '17 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.