# What would be an example of a really simple model with an intractable likelihood?

Approximate Bayesian computation is a really cool technique for fitting basically any stochastic model, intended for models where the likelihood is intractable (say, you can sample from the model if you fix the parameters but you cannot numerically, algorithmically or analytically calculate the likelihood). When introducing approximate Bayesian computation (ABC) to an audience it is nice to use some example model that is really simple but still somewhat interesting and that has an intractable likelihood.

What would be a good example of a really simple model that still has an intractable likelihood?

• Your socks example is really simple and mostly intractable... Dec 8, 2014 at 16:42
• Ps: The socks example link... Dec 8, 2014 at 16:51
• Well, I was hoping that the socks would be intractable, but you proved that it wasn't, right? :) Dec 8, 2014 at 16:54
• This is a good question! There are various toy examples in the literature but they feel a bit artificial to me. It would be nice to have a really simple model motivated by a real application with an intractable likelihood. I seem to remember seeing David Cox present something along these lines but I haven't seen it published... Dec 9, 2014 at 10:18
• This is an interesting question. Bounty for "beautiful" toy examples
– user340483
Oct 11, 2022 at 14:24

Two distributions that are used a lot in the literature are:

• The g-and-k distribution. This is defined by its quantile function (inverse cdf) but has an intractable density. Rayner and MacGillivray (2002) is a good overview of these, and one of many ABC papers which use it as a toy example is Drovandi and Pettitt (2011).
• Alpha stable distributions. These are defined by their characteristic function but have an intractable density except for a couple of special cases. This has applications in finance and is often used in sequential ABC papers, for example Yildirim et al (2013).
• The g-and-k distribution is a very good example where the quantile function is simple to express while the likelihood function is not available at all: $$Q(u;A,B,g,k)=A + B\left[1+c\dfrac{1-\exp\{-g\Phi(u)\}}{1+\exp\{-g\Phi(u)\}}\right]\{1+\Phi(u)^2\}^k\Phi(u)$$ The $\alpha$-stable distributions are less simple to explain to newbies. Dec 9, 2014 at 10:36
• Could someone add examples of situations one would model with these distributions? Nov 23, 2016 at 16:59

One example I came through a few weeks ago and quite like for its simplicity is the following one: given an original normal dataset $$x_1,\ldots,x_n\stackrel{\text{iid}}{\sim}\text{N}(\theta,\sigma^2)\,,$$ the reported data is (alas!) made of the two-dimensional summary $$S(x_1,\ldots,x_n)=(\text{med}(x_1,\ldots,x_n),\text{mad}(x_1,\ldots,x_n))\,,$$ which is not sufficient and which does not have a closed form joint density.

• Just because the joint density is complicated to write down does not mean it does not have a closed form! "Intractable" is starting to seem like a matter of opinion in this thread. Perhaps you could clear that up by explaining what you mean by "intractable"?
– whuber
Dec 8, 2014 at 17:24
• Since I do not know of anyone who can compute this density, I call it intractable ... Since I have no computer program that can produce the numerical value of this likelihood, I am forced to use an ABC algorithm. Dec 8, 2014 at 17:53
• ABC does not compute the likelihood but uses simulations from the data to provide a sample of parameters that is an approximation of the true posterior. At the end of the day/computation, I am not the wiser about the likelihood function and I cannot produce a numerical value for $L(\theta|x_1,\ldots,x_n)$. Dec 8, 2014 at 19:36
• @whuber If one could successfully compute the likelihood, the example would not be very suitable for demonstrating an algorithm for approximating posteriors without computing likelihood$\times$prior products. Dec 9, 2014 at 10:24
• @whuber I think your interpretation (2) in the comment beginning "What I am wondering" is at least essentially the intended one. However, I don't understand your last remark "unless your ABC algorithm is taking a long time to execute" - the point of the question is that the expensive likelihood evaluation will be avoided by using ABC instead. Dec 9, 2014 at 10:31