I am reading Kevin Murphy's explanation of importance sampling, and would like to test my understanding in three areas. The text states that importance sampling samples from any proposal $q(x)$, and then uses the samples the integral as described per below. I have three questions:

  • Am I correct to understand that $f(x^s)$ is sampled following the distribution $q(x)$?
  • Why can the second term of $var_q$ be dropped? Is it because the expectation with regards to q is defined as $\int p(x)f(x)q(x)dx$, and hence division by q(x) leaves a term that is indepedent of $q(x)$?
  • Why is minimizing the variance the de-facto natural choice? It definitely sounds desirable, but are there no other choices (e.g. that reduce bias, increase accuracy, etc.)?

enter image description here


1 Answer 1


Answering your questions in order:

1.) $f(x^s)$ is the function $f$ evaluated at the points sampled from $q$. So not technically. The random numbers $f(x^s)$ are not sampled from $q$; the $x$s are. Then you use those random numbers as inputs to the function $f$.

2.) Observe that $I = E_q[f(x)w(x)] = \int f(x) \frac{p(x)}{q(x)}q(x) dx = \int f(x) p(x)dx$. so it's why they say it is; This term doesn't have a $q$ in it, so it doesn't depend on that random data you're sampling.

3.) Yeah, good question. I think I remember it has something to do with the delta method.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.