Suppose we have $$ X_1, \ldots, X_n \mid \theta \, \mathop{\sim}^{iid} \, L(\cdot \mid \theta), \quad \theta \sim \pi $$ By Bayes' theorem, the corresponding posterior distribution is $$ \pi_n(\mathrm d \theta \mid X_{1:n}) \propto L_n(X_{1:n} \mid \theta) \pi(\mathrm d \theta) $$ where $L_n(X_{1:n} \mid \theta) = \prod_{i=1}^n L(X_i \mid \theta)$.

Suppose that instead of considering $L(\cdot \mid \theta)$ we perform inference, we consider an approximation $\tilde L(\cdot \mid \theta)$ such that $$ d_{TV}\big(L(\cdot \mid \theta), \tilde L(\cdot \mid \theta)\big) \leq K(\theta) $$ where $d_{TV}$ is the Total Variation distance.

Let $\tilde \pi_n(\mathrm d \theta) \propto \tilde L_n(X_{1:n} \mid \theta) \pi(\mathrm d \theta)$ be the posterior distribution for this approximate model. I was wondering what is known on the relation between $\pi_n$ and $\tilde \pi_n$ and if there is some Lispchitz results (wrt to a suitable metric, possibly different from the TV one) to the likelihood function, or at least some bound on the distance between the two.

We can also assume that $\sup_{\theta} K(\theta)$ is bounded and, if it is more convenient, that $L(\cdot \mid \theta)$ is a distribution over the nonnegative integers.


1 Answer 1


The question of robustness of the posterior with respect to perturbations of prior/likelihood/model/data is crucial for Bayesian inference, but also very delicate, since things really depend on which metric(s) you consider, and there are positive and negative results.

I am not aware of any positive results if the likelihood perturbation is measured by total variation, on the contrary, there is a negative result in this paper by Owhadi, Scovel and Sullivan "On the Brittleness of Bayesian Inference".

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.