# Noisy (and biased) MCMC?

When estimating intractable expectations using MCMC, we usually assume we can evaluate the (unnormalized) target density exactly at any point. I.e. we're interested in evaluating expectations w.r.t. some distribution $\pi$, where $\pi(x) = \frac{1}{\mathcal{Z}} q(x)$, and we can query $q$ exactly and $\mathcal{Z}$ is not needed.

What if we can only get noisy estimates of $q(x)$, and what if these estimates have unknown bias? Can we say anything about the correctness of the resulting algorithm, or the bias of the resulting chain?

This arises if we would like to apply MCMC to problems where our target is itself an intractable expectation we can only estimate using MCMC "in the inner loop," i.e. when $$q(x) \equiv \mathbb{E}_{z \sim p}[f(z)] \approx \frac{1}{N}\sum_{t=1}^N f(z_t)$$ where $\{z_t\}$ are simulated from a Markov chain whose stationary distribution is $p$. For finite $N$, our "inner loop" estimates will be biased due to our choice of Markov kernel and initial distribution.

This is a bit different from what I've seen so far:

• The pseudo-marginal MCMC examples I've encountered assume you have access to unbiased estimates of $q(x)$
• In the "big data" setting, the likelihood function involves a large number of independent terms, so minibatch estimates of the full likelihood are unbiased. E.g. the references here: MCMC sampling with noisy likelihood

$$e[x_1 -> x_2]$$