2
$\begingroup$

For model selection within an ABC framework when the models are nested, say model 1 is equal to model 2 on some subset of the parameter space, is it better to try and do parameter inference or use a model selection framework (for example abcrf)?

In more detail; my specific case is that I only have two models $M_1$ and $M_2$. $M_1$ has an associated parameter space $\Theta_1\subset \mathbb{R}_{\geq 0}$, while for $M_2$ we have an associated parameter space $\Theta_2\subset \mathbb{R}_{\geq 0}^2$. $M_2$ is equal to $M_1$ along $(\theta_1,0)$. I have some data $D$ and wish to know which model is more appropriate. From here

1) Is it better to estimate $Pr(M_2|D)/Pr(M_1|D)$ or $Pr(\theta_2>0|D,M_2)/Pr(\theta_2=0|D,M_2)$?

2) If I cannot analytically obtain the likelihood and so use abc with some chosen summary statistics in place of $D$, does the answer to 1 change?

Regarding 1, I'll compare the options. Assuming $Pr(M_1|D)=Pr(M_2|D)$ we have \begin{align} \frac{Pr(M_2|D)}{Pr(M_1|D)}=\frac{\int_0^\infty \int_0^\infty Pr(D|M_2,(\theta_1,\theta_2))Pr((\theta_1,\theta_2|M_2))d\theta_1 d\theta_2}{ \int_0^\infty Pr(D|M_1,\theta_1)Pr(\theta_1|M_1)d\theta_1 } \end{align} For the numerator we notice \begin{align} \int_0^{\infty}Pr(D|M_2,(\theta_1,0))Pr((\theta_1,0)|M_2))d\theta_1 \\+ \int_0^\infty \int_{0^+}^\infty Pr(D|M_2,(\theta_1,\theta_2))Pr((\theta_1,\theta_2|M_2))d\theta_1 d\theta_2 \\ = Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) + Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2). \end{align} Concerning the denominator \begin{align} \int_0^\infty Pr(D|M_1,\theta_1)Pr(\theta_1|M_1)d\theta_1 \\ =Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) \end{align} Which results in \begin{align} \frac{Pr(M_2|D)}{Pr(M_1|D)}=\frac{ Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) + Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2)}{ Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2) }. \end{align} However the alternative is \begin{align} \frac{Pr(\theta_2>0|D,M_2)}{Pr(\theta_2=0|D,M_2)}=\frac{Pr(D|\theta_2>0,M_2)Pr(\theta_2>0|M_2)}{Pr(D|\theta_2=0,M_2)Pr(\theta_2=0|M_2)} \end{align} Have I made some error or is there some implicit assumption I've made that leads to this?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Expecting the estimate to be exactly equal to zero is unrealistic, especially for a continuous parameter space where the probability this happens is zero. This is why model comparison has been invented. (Note that I fail to see the role of ABC in this question.) $\endgroup$
    – Xi'an
    Dec 1, 2018 at 10:59
  • $\begingroup$ Thanks for your response. You're right, there's really two issues I have. The first is I don't know whether to (A) estimate the Bayes factor or (B) the posterior of $\theta_2$. The second is how the answer to the first issue translates into an abc setting. I've added a comparison of (A) vs (B), however that might have just confused matters. $\endgroup$
    – Michael
    Dec 3, 2018 at 16:15
  • 1
    $\begingroup$ Several of our papers on ABC have be centred on model comparison, as ABC can be used to approximate a posterior probability and hence a Bayes factor... $\endgroup$
    – Xi'an
    Dec 3, 2018 at 18:30
  • $\begingroup$ Yeah I think in the course of this discussion I've just convinced myself of the importance of those papers. Apologies if the above was unclear, but it was useful for me. $\endgroup$
    – Michael
    Dec 3, 2018 at 20:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.