Suppose we want a sample of size n from a truncated gaussian distribution with density

$f(x) = \dfrac{1}{\sqrt{2\pi}\sigma(1-\Phi((1-\mu )/\sigma ) }e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ , $x>1$


$p(x) = \dfrac{f(x)}{(1-\Phi((1-\mu )/\sigma )^{-1}}$

Then we can find an envelope for $p(x)$ which is equal to the density of the normal distribution with mean $\mu$ and variance $\sigma^2$.

For the rejection sampling method - just sample from a normal distribution with mean $\mu$ and variance $\sigma^2$ and throw away any sample $y$ that is less than or equal to one.

What is the difference between SIR and Rejection sampling in this case?

(SIR) - generate samples from $f$.

1.draw samples $x^1..x^N \sim h(x)$

2,calculate importance weights $w_i = f(x^i)/h(x^i)$

3,Normalize the weights as $q_i = w_i/\sum_j w_j$

4, resample from $\{x^1,..x^N \}$ where $y^i$ is drawn with probability $q_i$.

we can exchange $f(x^i)$ by $p(x^i)$ in 2 , since it will cancel in 3.

As in the rejection sampling we use the envelope $h(x) = p(x)$ and so we resample from the set in $4$, all with equal probabilities. But there are no approximation here since all draws are good that is all x in (4) come from a normal distribution.. and all that are less Than or equal to one we must throw away

Is my reasoning Wrong?

  • $\begingroup$ What do you mean by "all draws are good"? The justification for SIR is asymptotic, so your draws arein fact approximately distributed according to the target. $\endgroup$ – Taylor Jan 3 '18 at 0:03
  • $\begingroup$ I mean that any draw that we pick in 4 still come from a normal distribution and we keep only those that are greater than one and so the ones we keep come from an trunc normal distribution ...@Taylor $\endgroup$ – Danny Jan 3 '18 at 0:15
  • $\begingroup$ So why is that not exact $\endgroup$ – Danny Jan 3 '18 at 0:33
  • $\begingroup$ @Taylor: the justification for Monte Carlo is equally asymptotic, aka LLN. $\endgroup$ – Xi'an Jan 3 '18 at 4:28

In this special case, there is no difference: simulating a sample $x_1,\ldots,x_N$ from the $\mathcal{N}(\mu,\sigma^2)$ distribution will produce the same truncated sample $y_1,\ldots,y_n$ from the truncated Gaussian distribution because the importance weights are then zero or one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.