Robustness of Posterior distribution wrt likelihood function

Question

Suppose we have $$ X_1, \ldots, X_n \mid \theta \, \mathop{\sim}^{iid} \, L(\cdot \mid \theta), \quad \theta \sim \pi $$ By Bayes' theorem, the corresponding posterior distribution is $$ \pi_n(\mathrm d \theta \mid X_{1:n}) \propto L_n(X_{1:n} \mid \theta) \pi(\mathrm d \theta) $$ where $L_n(X_{1:n} \mid \theta) = \prod_{i=1}^n L(X_i \mid \theta)$.

Suppose that instead of considering $L(\cdot \mid \theta)$ we perform inference, we consider an approximation $\tilde L(\cdot \mid \theta)$ such that $$ d_{TV}\big(L(\cdot \mid \theta), \tilde L(\cdot \mid \theta)\big) \leq K(\theta) $$ where $d_{TV}$ is the Total Variation distance.

Let $\tilde \pi_n(\mathrm d \theta) \propto \tilde L_n(X_{1:n} \mid \theta) \pi(\mathrm d \theta)$ be the posterior distribution for this approximate model. I was wondering what is known on the relation between $\pi_n$ and $\tilde \pi_n$ and if there is some Lispchitz results (wrt to a suitable metric, possibly different from the TV one) to the likelihood function, or at least some bound on the distance between the two.

We can also assume that $\sup_{\theta} K(\theta)$ is bounded and, if it is more convenient, that $L(\cdot \mid \theta)$ is a distribution over the nonnegative integers.

Answer 1

The question of robustness of the posterior with respect to perturbations of prior/likelihood/model/data is crucial for Bayesian inference, but also very delicate, since things really depend on which metric(s) you consider, and there are positive and negative results.

I am not aware of any positive results if the likelihood perturbation is measured by total variation, on the contrary, there is a negative result in this paper by Owhadi, Scovel and Sullivan "On the Brittleness of Bayesian Inference".

Hope this helps.