# Deriving the bias of the self-normalizing importance sampling estimator

Suppose we are interested in the expectation of a test function $$f(X)$$ with respect to target distribution $$\pi(X) \propto \gamma(X)$$ using importance sampling with proposal distribution $$q(X)$$ with $$M$$ samples. Suppose we only know $$\gamma$$ and not $$\pi$$. Let $$w(X) = \frac{\gamma(X)}{q(X)}$$. Then we have

$$F = \mathbb{E}_{\pi}[f(X)] = \frac{\mathbb{E}_q[f(X)w(X)]}{\mathbb{E}_q[w(X)]} \approx \frac{\sum_{i=1}^M f(X_i)w(X_i)}{\sum_{j=1}^M w(X_j)} = \hat F.$$

How do I derive the bias of $$\hat F$$?

(below is one of my many attempts in finding a nice form for the expected value of $$\hat F$$) $$\mathbb{E}_q \Big[\frac{\sum_i^M f(X_i)w(X_i)}{\sum_j^M w(X_j)}\Big] = M \mathbb{E}_q \Big[ \frac{f(X_1)w(X_1)}{\sum_j w(X_j)}\Big] = M \int \frac{f(x_1)\gamma(x_1)}{\sum_{j=1}^M w(x_j)}\prod_{i=2}^M[ q(x_i) dx_i]$$

• There is no closed-form expression for the bias. Note that your integral term in the last equation involves the joint density of the $X_i$'s, not of a single $X$ (which should further be written $x$ in the integrand). Mar 30, 2020 at 6:42
• Is the updated integrand correct? Thanks for answering and pointing out the mistake.
– user
Mar 30, 2020 at 6:51
• Actually the product starts at $i=2$. Mar 30, 2020 at 10:52
• The asymptotic bias and variance can be derived using the delta method. Refer to pages 23--29 of Zabaras 2010
– jII
Sep 16, 2022 at 16:20