I am reading Kevin Murphy's explanation of importance sampling, and would like to test my understanding in three areas. The text states that importance sampling samples from any proposal $q(x)$, and then uses the samples the integral as described per below. I have three questions:
- Am I correct to understand that $f(x^s)$ is sampled following the distribution $q(x)$?
- Why can the second term of $var_q$ be dropped? Is it because the expectation with regards to q is defined as $\int p(x)f(x)q(x)dx$, and hence division by q(x) leaves a term that is indepedent of $q(x)$?
- Why is minimizing the variance the de-facto natural choice? It definitely sounds desirable, but are there no other choices (e.g. that reduce bias, increase accuracy, etc.)?