Is it possible to use an estimated Likelihood without summary statistics for MCMC sampling?

Question

I am currently trying to perform MCMC sampling using a (stochastic) model, for which I cannot derive a likelihood function, but which allows me to draw samples $y_\theta \sim p_{y|\theta}$, where $p_{y|\theta}$ is the distribution defined by the model with parameters $\theta$.

I already looked at several papers for likelihood-free parameter estimation, e.g. this one, but they usually use some kind of summary statistic, for which the likelihood is estimated (e.g. instead of estimating $L(\theta)=p_{y|\theta}(y_0|\theta)$ they may estimate $L_\mu(\theta)=p_{\mu|\theta}(\mu_0|\theta)$ where $\mu_0$ is the mean of the observed data). This supplementary likelihood can then be used for sampling.

For my current analysis, however, I cannot define any usable summary statistics. The reason is, that in addition to parameters my model also has some inputs, and I have very few samples for each distinct input. Instead, I hope to use the estimated likelihood directly.

So my approach would be the following:

1. For each update in the MCMC, draw a large number of samples $\hat{Y}_\theta=\{\hat{y}_{1,\theta},\ldots,\hat{y}_{N,\theta}\}$ from $p_{y|\theta}$

2. Estimate $\hat{p}_{y|\theta} \approx p_{y|\theta}$ using a kernel density estimate based on $\hat{Y}_\theta$.

3. Use this estimate to estimate the likelihood of the observed data as $\hat{L}(\theta) = \hat{p}_{y|\theta}(y_0|\theta)$

4. Accept or reject the update based on the estimated likelihood using the normal rules for MCMC (e.g. Metropolis).

However, I am unsure, what kind of bias I would introduce using this estimation. Looking at equation (35) and the section following it in the paper linked above, I would assume that this becomes equivalent to approximate Bayesian computation without using summary statistics given the choice of kernels mentioned in the text. Or is there some other bias I am introducing by not resorting to summary statistics? In general, I would assume, that the data itself can become its own summary statistic, but some kind of validation would be good.

Answer 1

Your approach would be equivalent to a standard ABC-MCMC method (see e.g. https://arxiv.org/abs/1802.09650) where the summary statistic is just the full data, i.e. $S(y) = y$.

The biases induced by this procedure are fairly standard, namely:

1. The approximation error of replacing the true posterior by the ABC posterior, and
2. The Monte Carlo error of using MCMC to approximate the posterior.