Using maximum Likelihood regression to get closer to the true posterior when doing Approximate Bayesian Computation : contradiction?

Question

Post-hoc adjustments are used to get closer to the true posterior distribution when doing Approximate Bayesian Computation. This is particularly important when using a rough algorithm like rejection/sampling. I have been using the multiple weighted linear regression as proposed by Beaumont et al (2002) for a while. This is particularly convenient since there is an analytic answer (very fast) to get the slopes and the intercepts. But as this linear model is somehow part of the ABC model, would it be more respectful (probably not the good word to use) to the whole Bayesian framework employed until that post-hoc adjustment to use a Bayesian regression model. That way, it could be possible to account for the Bayesian incertitude in the objective of getting closer to the true distribution.

I don't know exactly what to think about that, and the maximum Likelihood framework is good at getting closer to the true posterior distribution. Any line of thought or philosophical perspectives on that?

And What about considering the model used in the post-hoc operation as part of the ABC model? The same question could be extended to more recent post-hoc adjustments using more sophisticated machine learning algorithms (e.g., neural network and random forest).

Maybe there is too many questions here?

Answer 1

This is an interesting question I have also asked myself overtime. However, I came to the conclusion that the non-parametric Bayesian approach was not particularly appropriate or respectful or useful in this context.

The fundamental reason is that the learning process in ABC is mostly based on the prior predictive, not on the data. The algorithm produces a huge reference table of $(\theta_i,z_i)$'s from the prior predictive and infer the connection between $\theta$ and $z$ from this table before applying the construct to the actual observation $z^\text{obs}$. This is thus a context where [machine] learning techniques [like neural nets and random forests] and cross-validation make more sense than Bayesian inference on $\mathbb{E}[\theta|z]$.