Another question from Introducing Monte Carlo Methods with R by Robert and Casella.

Exercise 3.6 basically says the following. Suppose $f$ and $g$ are densities. Draw a random sample $X_1, \ldots, X_n \sim g$, then draw $(w_i \mid X_i) \sim Poisson( (f(X_i)/g(X_i) )$ for $i = 1, 2, \ldots, n$. Also define $X^*_i \mid w_i$ as iid draws from the categorical distribution $(X^*_i \mid w_i) \sim Categorical(\frac{w_1}{\sum_j w_j}, \ldots, \frac{w_n}{\sum_j w_j})$ for all $i$.

Based on the fact (which I've managed to show) that

$$ E_g\Big(w_i h(X_i)\Big)= E_f\Big(h(X_i)\Big),$$

we're supposed to "deduce that this sampling mechanism is marginally distributed from $f$". In other words, $X^{*}_{i} \sim f$.

I'm stuck. What I've tried is (pretend there is an $i$ subscript everywhere):

$$ \begin{aligned} \pi(x^*) &= \int_{-\infty}^{\infty} \sum_{w=0}^{\infty} \pi(x^* , w, x) dx\\ &= \int_{-\infty}^{\infty} \sum_{w=0}^{\infty} \pi(x)\pi(w \mid x)\pi(x^* \mid w , x) dx \\ &= \int_{-\infty}^{\infty} \sum_{w=0}^{\infty} g(x)\frac{\exp\Big(-\frac{f(x)}{g(x)}\Big) \Big(\frac{f(x)}{g(x)}\Big)^w}{w!} \frac{w}{\sum_j w_j} dx \end{aligned} $$

The sum is zero for $w=0$, so we change the bounds on the sum. After some manipulation, we get

$$ \begin{aligned} \pi(x^*) &= \int_{-\infty}^{\infty} \sum_{w=1}^{\infty} f(x)\frac{\exp\Big(-\frac{f(x)}{g(x)}\Big) \Big(\frac{f(x)}{g(x)}\Big)^{(w-1)}}{(w-1)!} \frac{1}{\sum_j w_j} dx \\ &= \int_{-\infty}^{\infty} \frac{f(x)}{\sum_j w_j} \sum_{w=1}^{\infty} \frac{\exp\Big(-\frac{f(x)}{g(x)}\Big) \Big(\frac{f(x)}{g(x)}\Big)^{(w-1)}}{(w-1)!}dx \\ &= \int_{-\infty}^{\infty} \frac{f(x)}{\sum_j w_j}dx \end{aligned} $$

which seems wrong to me. But I can't tell where I'm going wrong. Any help would be appreciated!

  • $\begingroup$ Given that this question is about one exercise you want help with, I would suggest you add the self-study tag. $\endgroup$ – Xi'an Jun 2 '17 at 19:48

Thank you for the question and sorry for being vague! The way the exercise is set is slightly different though:

Exercise 3.6. Given an importance sample $(X_i,f(X_i)/g(X_i))$, show that if $\omega_i$ has a Poisson distribution $\omega_i\sim\mathcal{P}(f(X_i)/g(X_i))$, the estimator $$ \frac{1}{n}\sum_{i=1}^n \omega_i h(x_i) $$ is unbiased. Deduce that the sample derived by this sampling mechanism is marginally distributed from $f$.

The first part of the question is about $$\mathbb{E}_g\Big(w_i h(X_i)\Big)= \mathbb{E}_f\Big(h(X_i)\Big)$$ as you gather. The second and vaguer part means that, if one takes the sample $$(\underbrace{X_1,\ldots,X_1}_\text{$\omega_1$ times},\ldots,\underbrace{X_n,\ldots,X_n}_\text{$\omega_n$ times})$$ then it behaves like a sample from $f$. Almost because $$\sum_{i=1}^n \omega_i \ne n$$ for most realisations,

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.