I am currently trying to perform MCMC sampling using a (stochastic) model, for which I cannot derive a likelihood function, but which allows me to draw samples $y_\theta \sim p_{y|\theta}$, where $p_{y|\theta}$ is the distribution defined by the model with parameters $\theta$.

I already looked at several papers for likelihood-free parameter estimation, e.g. this one, but they usually use some kind of summary statistic, for which the likelihood is estimated (e.g. instead of estimating $L(\theta)=p_{y|\theta}(y_0|\theta)$ they may estimate $L_\mu(\theta)=p_{\mu|\theta}(\mu_0|\theta)$ where $\mu_0$ is the mean of the observed data). This supplementary likelihood can then be used for sampling.

For my current analysis, however, I cannot define any usable summary statistics. The reason is, that in addition to parameters my model also has some inputs, and I have very few samples for each distinct input. Instead, I hope to use the estimated likelihood directly.

So my approach would be the following:

  1. For each update in the MCMC, draw a large number of samples $\hat{Y}_\theta=\{\hat{y}_{1,\theta},\ldots,\hat{y}_{N,\theta}\}$ from $p_{y|\theta}$

  2. Estimate $\hat{p}_{y|\theta} \approx p_{y|\theta}$ using a kernel density estimate based on $\hat{Y}_\theta$.

  3. Use this estimate to estimate the likelihood of the observed data as $\hat{L}(\theta) = \hat{p}_{y|\theta}(y_0|\theta)$

  4. Accept or reject the update based on the estimated likelihood using the normal rules for MCMC (e.g. Metropolis).

However, I am unsure, what kind of bias I would introduce using this estimation. Looking at equation (35) and the section following it in the paper linked above, I would assume that this becomes equivalent to approximate Bayesian computation without using summary statistics given the choice of kernels mentioned in the text. Or is there some other bias I am introducing by not resorting to summary statistics? In general, I would assume, that the data itself can become its own summary statistic, but some kind of validation would be good.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.