6
$\begingroup$

From what I understand, SIR is a mechanism for sampling from a distribution $p$ that works as follows:

  1. Approximate a target distribution $p$ using an importance sample $S$ from a proposal distribution $q$
  2. Draw a small sample $S_\text{small}$ from $S$ with replacement according to $p$

The result, $S_\text{small}$, is a sample from $p$.

  1. Why use a small subset and not the full sample $S$ in 1?
  2. Why is this a variance reduction technique, and why does it matter to reduce the variance?
Improve this question
$\endgroup$
4
  • $\begingroup$ The resulting sample $S_\text{small}$ is not a sample from $p$ due to the sampling from self-normalised weights. $\endgroup$
    – Xi'an
    Nov 25, 2019 at 6:14
  • $\begingroup$ @Xi'an would you please elaborate the reason why $S_{small}$ is not a set of samples from $p$? Thanks $\endgroup$
    – True
    Jul 12, 2021 at 15:31
  • 1
    $\begingroup$ @True: dividing the importance weights by the sum of the importance weights modifies or biases the distribution of the resulting sample. $\endgroup$
    – Xi'an
    Jul 12, 2021 at 17:24
  • $\begingroup$ @Xi'an I see. Thanks a lot! :) $\endgroup$
    – True
    Jul 13, 2021 at 20:56

1 Answer 1

Reset to default
2
$\begingroup$

The main point is that most of the time, SIS/SIR is used in a sequential setting and this is why one needs to reallocate particles to best deal with the next time integration. When performing IS, it is quite common that all of the weightings are attributed to only a very small subpart of the particles whereas they characterise the area of interest. Resampling allows reallocating particles from low-density regions into high-density regions making thus a more optimal use of our available particles (because the more particles, the more costly the procedure).

As I understand it, $S_\rm{small}$ is not smaller than $S$: they have typically the same size.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.