I would like to know if it is possible to do a model-selection under the approximate Bayesian computation paradigm and using particular sets of summary statistics (e.g., S1 and S2) that differ for each model (e.g., M1 and M2). Sets S1 and S2 are obtained from the same data (i.e., S1(D) and S2(D)) but the composition in summary statistics could differ between the two sets S1 and S2. The two models have different numbers of parameters. In other words, this is the worst setup for model-selection under approximate Bayesian computation.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
This is an interesting perspective which I have pondered for a while but I believe using different statistics for different models does not produce a coherent outcome. My reasoning is that, removing all errors due to Monte Carlo and ABC errors (infinite number of simulations and zero tolerance $\epsilon$) one would compare the marginal of $S_1$ under model $M_1$ with the marginal of $S_2$ under model $M_2$. These marginals are not comparable.
-
$\begingroup$ Using cross-validation under the ABC framework. First, we need to learn the parameters (theta1 | M1, S1, D) and (theta2 | M2, S2, D), where theta(s) are the parameter values specific to each model (M1, M2), S(s) the summaries also specific to each model and D a dataset. Then we want to compare the fit of M1 and M2 on a new dataset (e.g., D2) using theta1 and theta2 respectively. Could we use then S3 = {S1, S2} to compare the fit of the two models? The fit could be evaluated base on the distribution of the distances obtained when using S3, by drawing from the posterior predictive of M1 and M2? $\endgroup$– SimonLLCommented Oct 7, 2017 at 15:32
-
$\begingroup$ The two posteriors are based on different observables, $s_1$ and $s_2$, respectively, which makes comparing the models incoherent. $\endgroup$– Xi'anCommented Oct 8, 2017 at 20:19