What difference does it make working with a big or small dataset in ABC? Do we get any computational benefits by reducing a very big dataset when doing inference using ABC methods? My understanding is that the rejection criterion in ABC is related to how well we approximate the full likelihood of the dataset which is typically captured in some low-dimensional summary statistics vector. Thus, the computational difference in doing inference on a very big and a very small dataset is only reflected in the amount of computation required to compute the summary statistics.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
What difference does it make working with a big or small dataset in ABC?
It all depends on the structure of the dataset and the complexity of the model behind. In some settings the size of the data may be the reason for conducting an ABC inference as the likelihood takes too much time to compute. But there is no generic answer to the question since in the ultimate case when there exists a sufficient statistic of fixed dimension size does not matter (and of course ABC is unlikely to be needed).
Do we get any computational benefits by reducing a very big dataset when doing inference using ABC methods?
In most settings, ABC proceeds through a set of summary statistics that are of a much smaller dimension than the data. In that sense they are independent of the size of the data, except that to simulate values of the summaries, most models require simulations of the entire dataset first. Unless a proxy model is used as in synthetic likelihood.
...the rejection criterion in ABC is related to how well we approximate the full likelihood of the dataset which is typically captured in some low-dimensional summary statistics vector.
You have to realise that the rejection is relative to the distribution of the distances between the observed and the simulated summaries [simulated under the prior predictive], rather than absolute. In other words, there is no predetermined value for the tolerance. This comes in addition to the assessment being based on an insufficient statistics rather than the full data. This means that, for a given computing budget, the true likelihood of an accepted parameter may be quite low.
the computational difference in doing inference on a very big and a very small dataset is only reflected in the amount of computation required to compute the summary statistics
Again, it all depends on the choice of summary statistics: if one picks the first ten observations, the size of the dataset does not matter. If one picks summaries that can be simulated at once, the size of the dataset does not matter. But it is correct that in most practical settings the complexity of ABC grows with the sample size.
-
1$\begingroup$ Thanks for the great reply. I'm trying to draw an analogy with inference on tractable likelihood models using standard MCMC, where each iteration requires a full pass over the dataset, hence linearly scales with dataset size. Replacing the dataset with a "informative subset" of the data in that case would clearly speed up MCMC inference. As per my understanding, this is not the case in likelihood-free inference. $\endgroup$– DionCommented Sep 3, 2019 at 16:13
-
1$\begingroup$ There exist MCMC schemes like pseudo-marginal MCMC that are sublinear in the sample size, including some Gibbs implementations using conditionally sufficient statistics. ABC can also be sublinear if one decides to use statistics based on subsamples, including Gibbs versions. $\endgroup$– Xi'anCommented Sep 4, 2019 at 2:30