# Does approximating the likelihood function violate the likelihood principle in Bayesian Inference?

Suppose we have a prior $$p(\theta)$$ and a likelihood function $$L(\theta|x)$$, and that the likelihood $$L(\theta|x)$$ is intractable somehow (difficult or impossible to compute) and we instead replace it with an approximate likelihood $$\tilde{L}(\theta|x)$$ and use the approximate posterior $$\tilde{p}(\theta|x) \propto p(\theta) \tilde{L}(\theta|x)$$ to conduct inference (for example, calculate the posterior mean of $$\theta$$). Does this violate the likelihood principle?

Intuitively it seems to violate the likelihood principle because you're not conducting inference under the likelihood $$L(\theta|x)$$ anymore (though I'm not sure how to prove this formally). However, in reality don't you 'choose' (a fairly arbitrary) likelihood function anyway (because you don't actually know what the true data generating model is), so what's stopping you from calling the approximate likelihood $$\tilde{L}(\theta|x)$$ the 'real' or 'original' likelihood?

For example, suppose we know for a fact that we have data $$x$$ generated from binomial distribution $$L(n,p|x)=B(n,p)$$, and I choose some (subjective, not dependent on the data) prior $$p(n,p)$$ on the model parameters. Suppose however that I replace $$L(n,p|x)$$ with $$\tilde{L}(n,p|x)=\mathcal{N}(np,np(1-p))$$ (a normal distribution approximate, discretised if necessary if a continuous distribution is a problem) and conduct inference using the posterior $$\tilde{p}(n,p|x) \propto p(n,p) \tilde{L}(n,p|x)$$. Would I be violating the likelihood principle in this case?

Would I still be violating the likelihood principle in the above example, if I didn't know $$B(n,p)$$ was the true model?

• It does not violate the likelihood principle because under the hood, it does assume all information lies within the likelihood function. It’s just that we can’t compute it for various reasons. Apr 30, 2022 at 12:34
• The question is too vague to be answered, because it all depends on the construction of $\tilde L$. For instance, the Approximate Bayesian computation (ABC) approach does violate the likelihood principle since, in its vanilla version, it simulates pseudo-data from the sampling model. And/or uses insufficient statistics. Apr 30, 2022 at 12:45
• I found a related post. Apr 30, 2022 at 13:06
• I think I'm confused because I know the statistical model we're using is almost always wrong and not the true data generating model. Therefore, doesn't pretty much all models violate the likelihood principle unless you can be sure that your likelihood is the correct model? Unless I've misunderstood and the likelihood principle just say all inferences have to come from your assumed likelihood model, whether it's correct or not? In that case, why does ABC violate the likelihood principle since I can just argue I'm using a different likelihood (that is not intractable to compute)? Apr 30, 2022 at 14:25
• @Xi'an I have also added a simple example to the original post which hopefully explains my confusion. Apr 30, 2022 at 15:20

The likelihood principle states that two experiments with the same likelihood functions (up to a multiplicative constant) of the same parameter $$\theta$$, provide the same evidence on the parameter $$\theta$$.