1
$\begingroup$

We are interested in the value of $\mu = \int f(x)dx$, and we have a factorization of $f$ as $f(x) = h(x)p(x)$, where $p(x)$ is a density. The general way to apply importance sampling is to follow the following procedure:

1) Draw $m$ samples $x^{(i)}$ from a trial density $g(x)$

2) Calculate the weights $w^{i} = p(x^{i})/g(x^{i})$

3) Aquire the estimate $\hat{\mu} = \frac{ \sum w^{i}h(x^{i})}{(w^{1}+w^{2}+...+w^{m})}$

My question is regarding step 2. Why is it necessary to "reapply" $g$ to $x^{i}$, when $x^{i}$ is already generated from the distribution with density $g$? Why could it not just be calculated as $p(x^{i})/x^{i}$ ?

$\endgroup$

1 Answer 1

2
$\begingroup$

As described, you use the weights $w^i = p(x^i) / g(x^i)$. We're dividing by something here: when can $g(x) = 0$? In order for $g(x) = 0$ we need to have sampled an $x_0$ such that $g(x_0) = 0$. But the density at $x_0$ is exactly $0$, so the probability (under $g$) of the set $\{x : g(x) = 0\}$ is exactly $0$. This means with probability $1$ we will never see $g = 0$ in these weights, so we don't have to worry about this.

What if instead we use some other weighting like the one you propose where we divide by $x$? Maybe that would cancel things out in the integral, but we may easily have $P_g(X = 0) > 0$, in which case the weights have a positive probability of being infinite/undefined, or even if $P_g(X = 0)$ the ratio with $x$ in the denominator may be very poorly behaved. For example, in what ought to be one of the friendliest of cases, when $X$ is Gaussian $1 / X$ does not have a finite mean.

Furthermore, we aren't even trying to cancel out the $x$. We instead want to change what we're effectively taking the expectation of in the first place, so having a $\frac 1x$ term wouldn't even help with that.

We have $$ \frac 1m \sum_{i=1}^m x^{(i)} \to_p E_g(X) = \int_{\mathbb R} x g(x)\,\text dx. $$

If we instead just use $w^i \cdot h(x^i)$ we have $$ \frac 1m \sum_{i=1}^m \frac{p(x^i)}{g(x^i)} h(x^i) \to_p E_g\left[ \frac{p(X)}{g(X)} h(X)\right] = \int_{\mathbb R} p(x) h(x) \,\text dx = \mu. $$

We never had any inconvenient $x^{(i)}$s that we needed to cancel out anyway.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.