I want to estimate $p(Q|x)$. I have an observed distribution for $p(x)$ - i.e. samples, and I can compute $p(x|Q)$ using MC simulation (not analytically). I have no belief about $p(Q)$ so we can make it uniform. We can use Bayes rule to define $$ p(Q|x) = \frac{p(x|Q)~p(Q)}{p(x)} $$ So, I can sample from $p(Q)$ and run these through the simulation to get $p(x|Q)$, but I'm not sure how to incorporate $p(x)$ in the context of these samples / simulations. Should I literally to divide the probabilities for matched values of $x$?
2
-
4$\begingroup$ Are you sure it's $p(x)$, not $p(x|Q)$, that you have a parametric form of? Usually it's the integrated likelihood that is intractable, not the likelihood itself. Anyway, if it's really the likelihood you don't have in closed form, you may want to look at 'likelihood-free' Bayesian inference. $\endgroup$– DurdenJul 7 at 15:23
-
$\begingroup$ Thanks - I am sure it is $p(x)$. Here $x$ is observed data, which can be used to estimate $Q$ with nonzero bias and variance. I'm proposing a way to adjust the bias, but I want to quantify / illustrate how variance in $x$ is maintained as variance in the estimated $Q | x$. $\endgroup$– jessexknightJul 7 at 17:23
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
In case you did not follow the link provided by Durden, here is the basic approach to likelihood-free / approximate Bayesian computation (ABC) of $p(Q|x^\text{obs})$:
- Simulate $Q$ from the prior $p(Q)$
- Simulate $x$ from the sampling distribution $p(x|Q)$
- Accept $Q$ if $d(x,x^\text{obs})<\epsilon$ else reject
where the value of $\epsilon$ is chosen as small as possible while allowing for a sufficient number of acceptance in an acceptable time.
-
$\begingroup$ Thanks - I have edited the question to remove "parametric form for $p(x)$" to avoid this confusion $\endgroup$ Jul 7 at 17:28