Model selection: comparing Bayesian models with likelihood vs likelihood-free (Approximate Bayesian Computation)

Question

I have two families of models that can possibly explain the data at hand.

One family is rather process-based, using fairly complicated simulations and Approximate Bayesian Computation to estimate the unknown parameters of the process. If I were to remain in the ABC framework, I know how I can use different simulation models to perform model selection.

The other family of models is more descriptive and involves likelihood computations for Bayesian inference. Again, if I were to remain in this framework, I know I can compare models using Bayes factor and model posteriors for model selection.

However, I would need to compare the two families of models (ABC estimations versus non-ABC estimations), and in that case I am much less certain: the parameters do not describe the same things at all, and the estimation methods are different too. Does it make sense to compare models between ABC/non-ABC estimations using BIC or is it unsound?

I am not a statistician, so do not hesitate asking me to clarify, I will try my best!

Answer 1

Brain-storming ideas here (not 100% sure on either):

• There is a modern ABC method called SNLE (Sequential Neural Likelihood Estimation) - which will give you an approximate likelihood (an actual analytical form of it - either using Mixture Density Networks, or Normalizing Flows). You can then maybe compare that with the theoretical model using Bayes factor.

• Another possibility is to use the samples generated from the ABC, which are essentially samples from the posterior $p(\theta|x_{obs})$. Choose the mode of this posterior (e.g., using KDE or some other methods), which I can interpret as "this is the most likely parameter to have generated the data" - then feed it to the simulator to get many samples of $x_{i,sim}(\theta_{mode})$, which should be a representing samples of the underlying likelihood $p(x|\theta_{mode})$. Now you can maybe compare this ABC empirical likelihood to the theoretical likelihood given by your theoretical model (maybe using KDE and Bayes factor? Or maybe using KDE and comparing the resulting density to the theoretical model density).