I am trying to find out whether the variance of weighted importance sampling can be shown to be bounded when the original samples are bounded.

More specifically, say $X_1, \cdots, X_n\in\mathbb{R}$ are random variables generated following a sample distribution $f(X)$, and we are interested in finding the expected value of these samples $\mu_g = {\bf E}_g[X]$ under a different distribution $g(X)$. Define the importance sampling ratio to be $w_k = \frac{g(X_k)}{f(X_k)}$, which is always nonnegative. Then $\mu_g$ can be estimated using the weighted importance sampling (WIS) estimator, which is defined as follows:

$v_n = \frac{\sum_{i=1}^n w_i X_i}{\sum_{k=1}^n w_k} = \sum_{i=1}^n w_i^* X_i $,


$w_i^* = \frac{w_i}{\sum_{k=1}^n w_k}\le 1$.

Now, let us consider that $|X|<C$, where $C$ is a positive constant value. Then can we show that ${\bf Var}_l[v_n]$ is bounded, perhaps by showing that $v_n$ is bounded?

I tried to show $v_n$ is bounded. Here is my attempt:

$|v_n| = |\sum_{i=1}^n w_i^* X_i|\le \sum_{i=1}^n |w_i^* X_i| \le C\sum_{i=1}^n |w_i^*| = C\sum_{i=1}^n w_i^* = C$.

I am wondering if you can help me find a mistake in my argument.

  • $\begingroup$ Your post is old, but since nobody answered yet: you just derived a bound on the weighted importance sampling estimator, not its variance. $\endgroup$ – PThomasCS Sep 4 '15 at 21:26

You just need one further step: a bounded random variable has bounded variance.

Let X be a bounded variable, then the expectation EX and E X^2 are bounded, so Var(X) = E(X^2) - (EX)^2 is bounded, too.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.