2
$\begingroup$

I am pretty new to ABC stuff so I may be saying dumb things.

My question is: I ran an ABC with two models $M_1$ and $M_2$ and now I have an approximation of the posterior distribution for both model.

If I do a posterior check by re-runing enough simulation for which I sample the parameters from the posteriors, is it possible to get back the likelihood of both model in order to calculate their Bayes factor?

I was thinking (probably wrongly) that taking the ratio between the number of simulations falling under a small threshold $\epsilon$ may do something related to the Bayes factor, something that may be written like: $$ BF_{1,2}=\frac{P(M1 | d(M1,D) < \epsilon)}{P(M2 | d(M2,D) < \epsilon)}$$ (where $d()$ is the distance function used for the orginal ABC)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Since the Bayes factor is given by$$B_{12}(D) = \frac{\text{Pr}(M_1|D)}{\text{Pr}(M_2|D)}\Big/\frac{\text{Pr}(M_1)}{\text{Pr}(M_2)}$$the ratio of frequencies of simulations from $M_1$ and $M_2$ that are accepted need be divided by the prior probabilities of $M_1$ and $M_2$ if these reflect the number of times each model is simulated. Apart from this, the approximation is valid.

In a series of papers we pointed out the dangers of using summary statistics in this setting:

  1. Lack of confidence in approximate Bayesian computation model choice
  2. Relevant statistics for ABC model choice
Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thanks for your answer. Thus if both models are ran the same number of time this normalizing by the prior's ratio should'nt mater right? And thanks for the paper. I new 1 but not 2, and though my summary statistics raise a full world of problem and question that I may ask when I have more time! $\endgroup$
    – Simon C.
    Commented Jan 24, 2019 at 14:57
  • 1
    $\begingroup$ Yes, keeping the same number of simulations for both models avoids the correction by the prior weights, assuming both models take about the same time to simulate. $\endgroup$
    – Xi'an
    Commented Jan 24, 2019 at 15:48
  • 1
    $\begingroup$ interesting, why the time taken to simulate should enter into consideration here? What if the real data results from generative process acting at really large time scale? Could it be possible to say that, even if one model take much more time than the other (let say ~ 10 time in my case), they are both equal wrt the original process ( ~ 4 millions times)? (By the way: thank you for your answers, not only this one but all the one you give on Cross Validate they are an amazingly rich source of learning people like my. It's awesome to see that you take time to do so.) $\endgroup$
    – Simon C.
    Commented Jan 24, 2019 at 16:54
  • 1
    $\begingroup$ My remark is a simple one, namely that, to save computing time, the number of simulations from the most expensive model may be cut down with the ABC Bayes factor still available by the correction. Thanks for your nice comments! $\endgroup$
    – Xi'an
    Commented Jan 24, 2019 at 16:57
  • $\begingroup$ Oh right! In that case the prior would be useful, I see. Thanks again. $\endgroup$
    – Simon C.
    Commented Jan 24, 2019 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.