4
$\begingroup$

Suppose I want to sample the posterior distribution of a multivariate $\beta$ given some scalar $x$. By Bayes' theorem, this distribution is

$$P(\beta|x) \propto P(x|\beta)P(\beta) $$

I don't have the exact form of $P(x|\beta)$ but I can generate samples from it. I have the exact form of the prior $P(\beta)$ and can sample from it. How do I turn that into a sample from the posterior? Can I just multiply them together?

$\endgroup$
3
$\begingroup$

This situation - one where the likelihood cannot be evaluated, but can be sampled from - is a problem that ABC (Approximate Bayesian Computation) deals with. In general, attaining samples from the true posterior is likely not possible in this situation, but using various ABC techniques, we can draw samples that closely approximate the posterior.

One crude but easy to implement algorithm is just to use rejection sampling. Given your data $x$, we can define some summary statistic $T(x)$ of our data - it is more efficient for $T(x)$ to be sufficient, but this is not necessary. One choice of $T(x)$ could just be the mean of the data. Then, the algorithm is:

  1. Sample parameters $\beta \sim p(\beta)$
  2. Sample fake data conditioned on the parameters: $x^* \sim p(x^* \vert \beta)$
  3. Compare the summary statistic of our fake data with our real data - if they are "close enough", say within $\epsilon$ distance of each other, accept $\beta$ as a sample from our approximate posterior.

This is only one of the many ABC approaches that you could use for solving this problem.

$\endgroup$
1
  • $\begingroup$ You can use[abc] to search for entries on ABC in Cross Validated. $\endgroup$
    – Xi'an
    Jan 29 '20 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.