Importance sampling - derivation

Question

I am reading Kevin Murphy's explanation of importance sampling, and would like to test my understanding in three areas. The text states that importance sampling samples from any proposal $q(x)$, and then uses the samples the integral as described per below. I have three questions:

• Am I correct to understand that $f(x^s)$ is sampled following the distribution $q(x)$?
• Why can the second term of $var_q$ be dropped? Is it because the expectation with regards to q is defined as $\int p(x)f(x)q(x)dx$, and hence division by q(x) leaves a term that is indepedent of $q(x)$?
• Why is minimizing the variance the de-facto natural choice? It definitely sounds desirable, but are there no other choices (e.g. that reduce bias, increase accuracy, etc.)?

Answer 1

Answering your questions in order:

1.) $f(x^s)$ is the function $f$ evaluated at the points sampled from $q$. So not technically. The random numbers $f(x^s)$ are not sampled from $q$; the $x$s are. Then you use those random numbers as inputs to the function $f$.

2.) Observe that $I = E_q[f(x)w(x)] = \int f(x) \frac{p(x)}{q(x)}q(x) dx = \int f(x) p(x)dx$. so it's why they say it is; This term doesn't have a $q$ in it, so it doesn't depend on that random data you're sampling.

3.) Yeah, good question. I think I remember it has something to do with the delta method.