How do I compute the KL divergence between my MCMC samples and the target distribution?

Question

Assume $(E,\mathcal E,\lambda)$ is a $\sigma$-finite measure space and $\nu$ is a probability measure on $(E,\mathcal E)$ with $\nu\ll\lambda$. Furthermore, assume that $\mu=\sum_{i=0}^{n-1}\delta_{X_i}$, where $(X_n)_{n\in\mathbb N_0}$ is a Markov chain on $(E,\mathcal E)$ with invariant distribution $\nu$. Can we give a nice expression for the KL divergence $$H(\mu,\nu):=\begin{cases}\displaystyle\int\ln\frac{{\rm d}\mu}{{\rm d}\nu}\;{\rm d}\mu=\int\frac{{\rm d}\mu}{{\rm d}\nu}\ln\frac{{\rm d}\mu}{{\rm d}\nu}\;{\rm d}\nu&\text{, if }\mu\ll\nu;\\\infty&\text{, otherwise}\end{cases}$$ between $\mu$ and $\nu$? The idea is that I want to compute the KL divergence between samples $x_0,\ldots,x_{n-1}$ of a MCMC method, which correspond to realizations of $X_0,\ldots,X_{n-1}$, and the target distribution $\nu$. If necessarily, you can assume that $\lambda$ is the $d$-dimensional Lebesgue measure. In light of the ergodic theorem, we should be able to get an estimator for $H(\mu,\nu)$. But I struggle to see why we should have $\mu\ll\nu$ at all ...

Answer 1

As you note, there's no reason $\mu\ll \nu$ (or $\nu\ll \mu$) so the KL-divergence will typically not be finite/not exist.

You might need to look at other metrics such as total variation or Wasserstein distance