2
$\begingroup$

I had asked some similar questions in the past, but I never got either the answers or the discussion I was hopping for. So I will rephrase the problem to see if I can understand it myself.

I'm trying to fit a complex model to some data that take a large amount of time to run. I'm also unable to write down a Likelihood function to this problem and so I turned to approximate Bayesian computation (ABC).Now, given the slowness of my simulations, I used Sequential ABC (a strategy where the prior are updated at each iteration), as implemented in R EasyABC.

I had realised that the result posterior distribution is highly variable (Sequential ABC get stuck in "local minimum"). This lead me to tun ABC_sequential more than once, and from different prior. In the end I gather all simulation that were done and now I want to analysed this.

I also need to state two other facts:

  • I have very little knowledge to build the priors. In fact, contrary to the concept of Bayesian statistic (new knowledge updating old knowledge) I would like to remove all the influence of the priors from my estimates.

  • The resulting distributions had several peaks (given from the different runs of ABC_sequential).

I designed also my custom distance function and computed a simple rejection algorithm of the best 5% simulations. However, as is probably obvious by now, this "posterior distribution" is highly influenced by the non-uniform original distributions. How can I remove this influence?

A common suggestion is importance sampling. As far as I understand, this implies computing the ratio between the density in my posterior and my prior and use this as a weight for my simulations. I did this with R package densratio and the results are unsatisfactory... Do you have any suggestions, corrections, etc?

$\endgroup$
  • $\begingroup$ @Xi'an Can you elaborate? I did not understood what you mean by "reference probability measure". And isn't ABC simulation from prior? Thanks for the quick reply :) $\endgroup$ – Diogo Santos Apr 23 '18 at 10:35
  • $\begingroup$ So in the context of rejection sampling, it's impossible to remove the influence of the prior?!! Can you elaborate on the reason why the solution I tried is not correct? $\endgroup$ – Diogo Santos Apr 23 '18 at 10:39
  • $\begingroup$ @Xi'an Thanks for explaining the effect of merging the samples. What I keep wondering if there is really no way to remove the prior in this context... Because in theory, one could estimate or know the prior density and weight each point by it's inverse. Can you point me to some literature or explanation on why is this wrong? $\endgroup$ – Diogo Santos Apr 26 '18 at 8:05
  • $\begingroup$ Can you expand on that? Maybe on an answer, as that seems to answer my question! :) $\endgroup$ – Diogo Santos Apr 26 '18 at 13:24
  • 4
    $\begingroup$ If you don't like priors and want to go frequentist there's always indirect inference. (I like priors myself.) arxiv.org/abs/0908.0433 $\endgroup$ – Cyan Apr 29 '18 at 23:02
2
$\begingroup$

Given an intractable likelihood $f(x|\theta)$ of a generative model, associated with an observation $x^\text{obs}$, ABC produces simulations from the joint distribution $$\pi^\text{ABC}_{\epsilon}(\theta,z)\propto \pi(\theta)f(z|\theta) \mathbb{I}_{(0,\epsilon)}d(z,x^\text{obs})$$ or in the kernel generalisation (e.g., Fernhead and Prangle, 2012) $$\pi^\text{ABC}_{\epsilon}(\theta,z)\propto \pi(\theta)f(z|\theta) K(d(z,x^\text{obs})/\epsilon)$$ where $z$ denotes the pseudo-observation and $\epsilon$ the tolerance. There exist many ways of implementing this approach, but a core notion is that it always constitutes a simulation methodology on the parameter x pseudo-observation pair and hence that the parameter must be endowed with a probabilistic structure. Changing or updating the prior is of course feasible by different means, but I do not see a way to reinterpret the ABC output away from a Bayesian perspective.

For instance, reweighing each point $\theta_i$ of an ABC sample by the inverse of the prior density $\pi(\theta_i)^{-1}$ means replacing the initial prior $\pi(\cdot)$ with a flat (and possibly improper) prior. But not leaving the Bayesian paradigm.

As pointed out by Corey Yanofski in the above comment, a related resolution that does not involve a prior is indirect inference, where simulation is "only" used to find a closest value of the parameter.

$\endgroup$
  • $\begingroup$ Great answer! I'm particularly interested in the methodology you describe in your last paragraph. Why would that lead to an improper prior? As far as I understand, if I do something like that I would still compute a proper posterior then? Lately I've been looking to neural network method by Blum and François, 2010, that I think will not suffer from any of the problems described. In the end, my goal is to understand if there is a way to minimize the impact of having a non-uniform prior when there is no knowledge behind that. (I do understand the need for defining limits a priory). $\endgroup$ – Diogo Santos Apr 29 '18 at 18:19
  • 2
    $\begingroup$ When the parameter space is unbounded, a flat prior is improper. The posterior may still remain proper, though. When you mention using NNs as in Blum and François (2010), this means using a different kernel in the ABC acceptance, not changing the prior. NNs are one way of representing the intractable likelihood, having no relevance wrt the prior. $\endgroup$ – Xi'an Apr 29 '18 at 19:24
0
$\begingroup$

A related method which does not use a prior is history matching. Unlike ABC it is not an approximation to Bayesian inference. Instead it simply rules out parts of the parameter space whose simulations are never consistent with the observations. There's a recent publication in Statistical Science (McKinley et al 2018) comparing ABC and history matching.

$\endgroup$
  • 1
    $\begingroup$ But personally I prefer using priors too! You can try to specify weakly informative priors when you have limited information to base them on. $\endgroup$ – Dennis Prangle May 2 '18 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.