# Consistency of likelihood importance sampling estimator

In a lecture recently our lecturer described a method for approximating the expectation of a function over a posterior distribution using likelihood importance sampling. That is:

$$\mathbb{E}_{p(x|D)}[f(x)] \text{ can be approximated by } \frac{1}{\sum_{n=1}^{N}p(D|x^{(n)})}\sum_{n=1}^{N}p(D|x^{(n)})f(x^{(n)})$$

where $$x^{(n)}$$ are i.i.d. RV sampled from the prior $$p(x)$$, and $$p(D|x^{(n)})$$ is the likelihood of the observed data $$D$$.

He then claims that this estimator is biased but consistent. How can I see this? To see it is biased you usually find the expectation of the estimator and I cannot see how to do that here

$$\mathbb{E}[\frac{1}{\sum_{n=1}^{N}p(D|x^{(n)})}\sum_{n=1}^{N}p(D|x^{(n)})f(x^{(n)})]$$ as the denominator and numerator are non-independent functions of $$x$$. Am I missing something and the denominator can be considered a constant and taken out of the expectation?

Any help or intuition much appreciated.

• A sample from the prior need be weighted by the likelihood to become a sample from the posterior. That it is biased is due to the division by the average$$\frac{1}{N}\sum_{n=1}^{N}p(D|x^{(n)})$$whose expectation is the normalising constant (or marginal likelihood) and hence whose inverse does not have as expectation the inverse of the normalising constant. – Xi'an Oct 27 at 21:00