4
$\begingroup$

Suppose we are interested in the expectation of a test function $f(X)$ with respect to target distribution $\pi(X) \propto \gamma(X)$ using importance sampling with proposal distribution $q(X)$ with $M$ samples. Suppose we only know $\gamma$ and not $\pi$. Let $w(X) = \frac{\gamma(X)}{q(X)}$. Then we have

$$F = \mathbb{E}_{\pi}[f(X)] = \frac{\mathbb{E}_q[f(X)w(X)]}{\mathbb{E}_q[w(X)]} \approx \frac{\sum_{i=1}^M f(X_i)w(X_i)}{\sum_{j=1}^M w(X_j)} = \hat F.$$

How do I derive the bias of $\hat F$?

(below is one of my many attempts in finding a nice form for the expected value of $\hat F$) $$\mathbb{E}_q \Big[\frac{\sum_i^M f(X_i)w(X_i)}{\sum_j^M w(X_j)}\Big] = M \mathbb{E}_q \Big[ \frac{f(X_1)w(X_1)}{\sum_j w(X_j)}\Big] = M \int \frac{f(x_1)\gamma(x_1)}{\sum_{j=1}^M w(x_j)}\prod_{i=2}^M[ q(x_i) dx_i] $$

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ There is no closed-form expression for the bias. Note that your integral term in the last equation involves the joint density of the $X_i$'s, not of a single $X$ (which should further be written $x$ in the integrand). $\endgroup$
    – Xi'an
    Mar 30, 2020 at 6:42
  • $\begingroup$ Is the updated integrand correct? Thanks for answering and pointing out the mistake. $\endgroup$
    – user
    Mar 30, 2020 at 6:51
  • 1
    $\begingroup$ Actually the product starts at $i=2$. $\endgroup$
    – Xi'an
    Mar 30, 2020 at 10:52
  • 1
    $\begingroup$ The asymptotic bias and variance can be derived using the delta method. Refer to pages 23--29 of Zabaras 2010 $\endgroup$
    – jII
    Sep 16, 2022 at 16:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.