Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I would like to ask a quite vague question, in order to gather references and ideas about that topic.
My question is the following:
In cases where the computation of Likelihood in Bayesian framework is complex or computational inneficient, what other methods/approximations exist in order to either calculate/approximate the Likelihood or take samples from it.
There are two main ways (that I am aware of) of dealing with this problem when the likelihood is difficult to work with.
The (probably) more popular method is Approximate Bayesian Computation. Suppose I have observed data $x$ and want to infer parameters $\theta$. The basic idea behind this is to generate samples from an appropriate probability distribution $x_{\text{synthetic}} \mid \theta \sim\text{model}(\theta)$. If $x_{\text{synthetic}}$ is ''close'' to $x$ retain $\theta$. wikipedia page for ABC. This is okay if we can't write down the likelihood but can easily simulate from the model. (e.g. lots of predator-prey or birth-death type models).
An other method is to use a Gaussian Process surrogate model (emulator) - a fast approximation to the 'true' model. Here we basically construct $\widehat{\text{model}}(\theta)$ and base inferences on a fast, approximate model with nice statistical properties. A key article on the approach is Kennedy & O'Hagan 2001. Although this article is about calibrating a deterministic model, we can also construct stochastic surrogate models, e.g. Binois et al 2018 and use this for calibration/inference. The nice thing about the emulator approach is that we can choose to either emulate the likelihood function or construct an emulator for the model directly.
