# Variance of weighted importance sampling when random variables are bounded

I am trying to find out whether the variance of weighted importance sampling can be shown to be bounded when the original samples are bounded.

More specifically, say $X_1, \cdots, X_n\in\mathbb{R}$ are random variables generated following a sample distribution $f(X)$, and we are interested in finding the expected value of these samples $\mu_g = {\bf E}_g[X]$ under a different distribution $g(X)$. Define the importance sampling ratio to be $w_k = \frac{g(X_k)}{f(X_k)}$, which is always nonnegative. Then $\mu_g$ can be estimated using the weighted importance sampling (WIS) estimator, which is defined as follows:

$v_n = \frac{\sum_{i=1}^n w_i X_i}{\sum_{k=1}^n w_k} = \sum_{i=1}^n w_i^* X_i$,

where

$w_i^* = \frac{w_i}{\sum_{k=1}^n w_k}\le 1$.

Now, let us consider that $|X|<C$, where $C$ is a positive constant value. Then can we show that ${\bf Var}_l[v_n]$ is bounded, perhaps by showing that $v_n$ is bounded?

I tried to show $v_n$ is bounded. Here is my attempt:

$|v_n| = |\sum_{i=1}^n w_i^* X_i|\le \sum_{i=1}^n |w_i^* X_i| \le C\sum_{i=1}^n |w_i^*| = C\sum_{i=1}^n w_i^* = C$.

I am wondering if you can help me find a mistake in my argument.

• Your post is old, but since nobody answered yet: you just derived a bound on the weighted importance sampling estimator, not its variance. – PThomasCS Sep 4 '15 at 21:26