Suppose we are interested in the expectation of a test function $f(X)$ with respect to target distribution $\pi(X) \propto \gamma(X)$ using importance sampling with proposal distribution $q(X)$ with $M$ samples. Suppose we only know $\gamma$ and not $\pi$. Let $w(X) = \frac{\gamma(X)}{q(X)}$. Then we have

$$F = \mathbb{E}_{\pi}[f(X)] = \frac{\mathbb{E}_q[f(X)w(X)]}{\mathbb{E}_q[w(X)]} \approx \frac{\sum_{i=1}^M f(X_i)w(X_i)}{\sum_{j=1}^M w(X_j)} = \hat F.$$

How do I derive the bias of $\hat F$?

(below is one of my many attempts in finding a nice form for the expected value of $\hat F$) $$\mathbb{E}_q \Big[\frac{\sum_i^M f(X_i)w(X_i)}{\sum_j^M w(X_j)}\Big] = M \mathbb{E}_q \Big[ \frac{f(X_1)w(X_1)}{\sum_j w(X_j)}\Big] = M \int \frac{f(x_1)\gamma(x_1)}{\sum_{j=1}^M w(x_j)}\prod_{i=2}^M[ q(x_i) dx_i] $$

  • 1
    $\begingroup$ There is no closed-form expression for the bias. Note that your integral term in the last equation involves the joint density of the $X_i$'s, not of a single $X$ (which should further be written $x$ in the integrand). $\endgroup$
    – Xi'an
    Mar 30, 2020 at 6:42
  • $\begingroup$ Is the updated integrand correct? Thanks for answering and pointing out the mistake. $\endgroup$
    – user
    Mar 30, 2020 at 6:51
  • 1
    $\begingroup$ Actually the product starts at $i=2$. $\endgroup$
    – Xi'an
    Mar 30, 2020 at 10:52
  • 1
    $\begingroup$ The asymptotic bias and variance can be derived using the delta method. Refer to pages 23--29 of Zabaras 2010 $\endgroup$
    – jII
    Sep 16, 2022 at 16:20


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.