2
$\begingroup$

I am currently trying to perform MCMC sampling using a (stochastic) model, for which I cannot derive a likelihood function, but which allows me to draw samples $y_\theta \sim p_{y|\theta}$, where $p_{y|\theta}$ is the distribution defined by the model with parameters $\theta$.

I already looked at several papers for likelihood-free parameter estimation, e.g. this one, but they usually use some kind of summary statistic, for which the likelihood is estimated (e.g. instead of estimating $L(\theta)=p_{y|\theta}(y_0|\theta)$ they may estimate $L_\mu(\theta)=p_{\mu|\theta}(\mu_0|\theta)$ where $\mu_0$ is the mean of the observed data). This supplementary likelihood can then be used for sampling.

For my current analysis, however, I cannot define any usable summary statistics. The reason is, that in addition to parameters my model also has some inputs, and I have very few samples for each distinct input. Instead, I hope to use the estimated likelihood directly.

So my approach would be the following:

  1. For each update in the MCMC, draw a large number of samples $\hat{Y}_\theta=\{\hat{y}_{1,\theta},\ldots,\hat{y}_{N,\theta}\}$ from $p_{y|\theta}$

  2. Estimate $\hat{p}_{y|\theta} \approx p_{y|\theta}$ using a kernel density estimate based on $\hat{Y}_\theta$.

  3. Use this estimate to estimate the likelihood of the observed data as $\hat{L}(\theta) = \hat{p}_{y|\theta}(y_0|\theta)$

  4. Accept or reject the update based on the estimated likelihood using the normal rules for MCMC (e.g. Metropolis).

However, I am unsure, what kind of bias I would introduce using this estimation. Looking at equation (35) and the section following it in the paper linked above, I would assume that this becomes equivalent to approximate Bayesian computation without using summary statistics given the choice of kernels mentioned in the text. Or is there some other bias I am introducing by not resorting to summary statistics? In general, I would assume, that the data itself can become its own summary statistic, but some kind of validation would be good.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.