Text: Computational Statistics 2E by Givens and Hoetings

Section: Weight Degeneracy, Rejuvenation, and Effective Sample Size

I am having trouble following another result in the text. Below is a screenshot.

enter image description here

I am not sure on how $X$ is to be generated. I believe there are two options: 1) $X$ is generated using Sampling Importance Resampling, and 2) $X$ is generated using Sequential Importance Sampling.

I have ruled out option 2 because there is no notion of time in the equations (i.e. no subscript for $t$).

So I am assuming that there is a sample $X_1, \ldots, X_n$ ~ (approximately) iid from $f$ (the target). Let $g$ be an envelope for $f$.

Then the standardized importance weights are defined as $w(X) = \frac{f(x)/g(x)}{\sum_{i=1}^n f(x_i)/g(x_i)}$.

The authors are claiming that $E(w(X)) = \frac1n$, but this is not clear to me how to arrive there.

I have managed to put together something that is possibly sensible.

We know, from the definition of $w(X)$, that $\sum_{i=1}^n w(X_i) = 1$.

Then, $E \left( \sum_{i=1}^n w(X_i) \right) = 1$.

Since $E \left( \sum_{i=1}^n w(X_i) \right) = \sum_{i=1}^n E \left( w(X_i) \right)$ and there is and (approximate) iid sample we get that $$1 = \sum_{i=1}^n E \left( w(X_i) \right) = n \cdot E \left( w(X_i) \right)$$

My issue is that the authors don't explicitly state how $X$ is generated. Any insights would be helpful.


It turns out that my idea was in the right direction, however my implementation was incorrect. The solution is as follows:

Suppose $f$ is the target distribution and $g$ is the envelope for $f$. Using the SIR method sample $$X_1, \ldots, X_n \overset{\text{iid}}{\sim} g,$$ with importance weights defined by $w^*(X) := w_i^* = \frac{f(x_i)}{g(x_i)}$. Define the standardized importance weights by $w(X) := w_i = \frac{w_i^*}{ \sum_{i=1}^n w_i^* }.$

Then it follows that $\sum_{i = 1}^n w_i = 1,$ and hence we have that $E \left( \sum_{i = 1}^n w_i \right) = \sum_{i = 1}^n E(w_i)= 1$.

Now, although each $w_i$ is NOT independent, they are identically distributed. Hence, $\sum_{i = 1}^n E(w_i) = n E(w_i) = 1$. Therefore we obtain the desired result $$E(w_i) = \frac1n$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.