# Noisy (and biased) MCMC?

When estimating intractable expectations using MCMC, we usually assume we can evaluate the (unnormalized) target density exactly at any point. I.e. we're interested in evaluating expectations w.r.t. some distribution $\pi$, where $\pi(x) = \frac{1}{\mathcal{Z}} q(x)$, and we can query $q$ exactly and $\mathcal{Z}$ is not needed.

What if we can only get noisy estimates of $q(x)$, and what if these estimates have unknown bias? Can we say anything about the correctness of the resulting algorithm, or the bias of the resulting chain?

This arises if we would like to apply MCMC to problems where our target is itself an intractable expectation we can only estimate using MCMC "in the inner loop," i.e. when $$q(x) \equiv \mathbb{E}_{z \sim p}[f(z)] \approx \frac{1}{N}\sum_{t=1}^N f(z_t)$$ where $\{z_t\}$ are simulated from a Markov chain whose stationary distribution is $p$. For finite $N$, our "inner loop" estimates will be biased due to our choice of Markov kernel and initial distribution.

This is a bit different from what I've seen so far:

• The pseudo-marginal MCMC examples I've encountered assume you have access to unbiased estimates of $q(x)$
• In the "big data" setting, the likelihood function involves a large number of independent terms, so minibatch estimates of the full likelihood are unbiased. E.g. the references here: MCMC sampling with noisy likelihood

## 1 Answer

As you note, the pseudo-marginal theorem says that an MCMC will average out a noisy unbiased likelihood.

This is because an MCMC will converge exactly if the expectation value for transition rate

$$e[x_1 -> x_2]$$

based on the noisy likelihood is the same as for the exact likelihood.

It's actually very easy to say what will happen if you have a bias in your likelihood estimator - the MCMC will converge to the biased likelihood, minus the noise, due to the same argument as above ;)