2
$\begingroup$

For model selection within an ABC framework when the models are nested, say model 1 is equal to model 2 on some subset of the parameter space, is it better to try and do parameter inference or use a model selection framework (for example abcrf)?

In more detail; my specific case is that I only have two models $M_1$ and $M_2$. $M_1$ has an associated parameter space $\Theta_1\subset \mathbb{R}_{\geq 0}$, while for $M_2$ we have an associated parameter space $\Theta_2\subset \mathbb{R}_{\geq 0}^2$. $M_2$ is equal to $M_1$ along $(\theta_1,0)$. I have some data $D$ and wish to know which model is more appropriate. From here

1) Is it better to estimate $Pr(M_2|D)/Pr(M_1|D)$ or $Pr(\theta_2>0|D,M_2)/Pr(\theta_2=0|D,M_2)$?

2) If I cannot analytically obtain the likelihood and so use abc with some chosen summary statistics in place of $D$, does the answer to 1 change?

Regarding 1, I'll compare the options. Assuming $Pr(M_1|D)=Pr(M_2|D)$ we have \begin{align} \frac{Pr(M_2|D)}{Pr(M_1|D)}=\frac{\int_0^\infty \int_0^\infty Pr(D|M_2,(\theta_1,\theta_2))Pr((\theta_1,\theta_2|M_2))d\theta_1 d\theta_2}{ \int_0^\infty Pr(D|M_1,\theta_1)Pr(\theta_1|M_1)d\theta_1 } \end{align} For the numerator we notice \begin{align} \int_0^{\infty}Pr(D|M_2,(\theta_1,0))Pr((\theta_1,0)|M_2))d\theta_1 \\+ \int_0^\infty \int_{0^+}^\infty Pr(D|M_2,(\theta_1,\theta_2))Pr((\theta_1,\theta_2|M_2))d\theta_1 d\theta_2 \\ = Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) + Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2). \end{align} Concerning the denominator \begin{align} \int_0^\infty Pr(D|M_1,\theta_1)Pr(\theta_1|M_1)d\theta_1 \\ =Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) \end{align} Which results in \begin{align} \frac{Pr(M_2|D)}{Pr(M_1|D)}=\frac{ Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) + Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2)}{ Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) }. \end{align} However the alternative is \begin{align} \frac{Pr(\theta_2>0|D,M_2)}{Pr(\theta_2=0|D,M_2)}=\frac{Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2)}{Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2)} \end{align} Have I made some error or is there some implicit assumption I've made that leads to this?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Expecting the estimate to be exactly equal to zero is unrealistic, especially for a continuous parameter space where the probability this happens is zero. This is why model comparison has been invented. (Note that I fail to see the role of ABC in this question.) $\endgroup$
    – Xi'an
    Dec 1 '18 at 10:59
  • $\begingroup$ Thanks for your response. You're right, there's really two issues I have. The first is I don't know whether to (A) estimate the Bayes factor or (B) the posterior of $\theta_2$. The second is how the answer to the first issue translates into an abc setting. I've added a comparison of (A) vs (B), however that might have just confused matters. $\endgroup$
    – Michael
    Dec 3 '18 at 16:15
  • 1
    $\begingroup$ Several of our papers on ABC have be centred on model comparison, as ABC can be used to approximate a posterior probability and hence a Bayes factor... $\endgroup$
    – Xi'an
    Dec 3 '18 at 18:30
  • $\begingroup$ Yeah I think in the course of this discussion I've just convinced myself of the importance of those papers. Apologies if the above was unclear, but it was useful for me. $\endgroup$
    – Michael
    Dec 3 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.