22
$\begingroup$

Approximate Bayesian computation is a really cool technique for fitting basically any stochastic model, intended for models where the likelihood is intractable (say, you can sample from the model if you fix the parameters but you cannot numerically, algorithmically or analytically calculate the likelihood). When introducing approximate Bayesian computation (ABC) to an audience it is nice to use some example model that is really simple but still somewhat interesting and that has an intractable likelihood.

What would be a good example of a really simple model that still has an intractable likelihood?

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Your socks example is really simple and mostly intractable... $\endgroup$
    – Xi'an
    Commented Dec 8, 2014 at 16:42
  • 3
    $\begingroup$ Ps: The socks example link... $\endgroup$
    – Xi'an
    Commented Dec 8, 2014 at 16:51
  • $\begingroup$ Well, I was hoping that the socks would be intractable, but you proved that it wasn't, right? :) $\endgroup$
    – Rasmus Bååth
    Commented Dec 8, 2014 at 16:54
  • 4
    $\begingroup$ This is a good question! There are various toy examples in the literature but they feel a bit artificial to me. It would be nice to have a really simple model motivated by a real application with an intractable likelihood. I seem to remember seeing David Cox present something along these lines but I haven't seen it published... $\endgroup$
    – Dennis Prangle
    Commented Dec 9, 2014 at 10:18
  • 1
    $\begingroup$ This is an interesting question. Bounty for "beautiful" toy examples $\endgroup$
    – user340483
    Commented Oct 11, 2022 at 14:24

2 Answers 2

Reset to default
16
$\begingroup$

Two distributions that are used a lot in the literature are:

  • The g-and-k distribution. This is defined by its quantile function (inverse cdf) but has an intractable density. Rayner and MacGillivray (2002) is a good overview of these, and one of many ABC papers which use it as a toy example is Drovandi and Pettitt (2011).
  • Alpha stable distributions. These are defined by their characteristic function but have an intractable density except for a couple of special cases. This has applications in finance and is often used in sequential ABC papers, for example Yildirim et al (2013).
Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ The g-and-k distribution is a very good example where the quantile function is simple to express while the likelihood function is not available at all: $$Q(u;A,B,g,k)=A + B\left[1+c\dfrac{1-\exp\{-g\Phi(u)\}}{1+\exp\{-g\Phi(u)\}}\right]\{1+\Phi(u)^2\}^k\Phi(u)$$ The $\alpha$-stable distributions are less simple to explain to newbies. $\endgroup$
    – Xi'an
    Commented Dec 9, 2014 at 10:36
  • 5
    $\begingroup$ Could someone add examples of situations one would model with these distributions? $\endgroup$
    – conjectures
    Commented Nov 23, 2016 at 16:59
9
+150
$\begingroup$

One example I came through a few weeks ago and quite like for its simplicity is the following one: given an original normal dataset $$ x_1,\ldots,x_n\stackrel{\text{iid}}{\sim}\text{N}(\theta,\sigma^2)\,, $$ the reported data is (alas!) made of the two-dimensional summary $$ S(x_1,\ldots,x_n)=(\text{med}(x_1,\ldots,x_n),\text{mad}(x_1,\ldots,x_n))\,, $$ which is not sufficient and which does not have a closed form joint density.

Improve this answer
$\endgroup$
7
  • 3
    $\begingroup$ Just because the joint density is complicated to write down does not mean it does not have a closed form! "Intractable" is starting to seem like a matter of opinion in this thread. Perhaps you could clear that up by explaining what you mean by "intractable"? $\endgroup$
    – whuber ♦
    Commented Dec 8, 2014 at 17:24
  • 2
    $\begingroup$ Since I do not know of anyone who can compute this density, I call it intractable ... Since I have no computer program that can produce the numerical value of this likelihood, I am forced to use an ABC algorithm. $\endgroup$
    – Xi'an
    Commented Dec 8, 2014 at 17:53
  • 3
    $\begingroup$ ABC does not compute the likelihood but uses simulations from the data to provide a sample of parameters that is an approximation of the true posterior. At the end of the day/computation, I am not the wiser about the likelihood function and I cannot produce a numerical value for $L(\theta|x_1,\ldots,x_n)$. $\endgroup$
    – Xi'an
    Commented Dec 8, 2014 at 19:36
  • 3
    $\begingroup$ @whuber If one could successfully compute the likelihood, the example would not be very suitable for demonstrating an algorithm for approximating posteriors without computing likelihood$\times$prior products. $\endgroup$
    – Juho Kokkala
    Commented Dec 9, 2014 at 10:24
  • 5
    $\begingroup$ @whuber I think your interpretation (2) in the comment beginning "What I am wondering" is at least essentially the intended one. However, I don't understand your last remark "unless your ABC algorithm is taking a long time to execute" - the point of the question is that the expensive likelihood evaluation will be avoided by using ABC instead. $\endgroup$
    – Juho Kokkala
    Commented Dec 9, 2014 at 10:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.