6
$\begingroup$

I was just wondering-- is it possible/practical to apply bayes' theorem without an analytical expression for the prior, only samples?

For example, say you have sufficient draws from a posterior distribution from a previous experiment via MCMC methods, and you'd like to use that posterior as the prior for a new one. You have an analytical expression for the likelihood as before, but now only samples from the (new) prior. How would you proceed?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
8
$\begingroup$

The short answer is yes. Have a look at sequential MCMC/ particle filters.

Essentially, your prior consists of a bunch of particles ($M$). So to sample from your prior, just select a particle with probability $1/M$. Since each particle has equal probability of being chosen, this term disappears in the M-H ratio.

A big problem with particle filters is particle degeneracy. This happens because you are trying to represent a continuous distribution with discrete particles - there's no such thing as a free lunch!

Clarification for Srikant Vadali

The question as I read it is: I have output, i.e. posterior from a MCMC scheme. I want to use this posterior as a prior for a new data set.

This (probably) means that you have a discrete representation of a continuous distribution, i.e. a particle representation. So rather than doing a random walk on a continous distribution (say), you need to pick values from your prior, i.e. you pick a particle.

Toni et al., use this idea with ABC.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I am not sure I follow the suggestion properly. How does the idea of a particle filter help here? $\endgroup$
    – user28
    Nov 8, 2010 at 19:50
  • $\begingroup$ @Srikant-vadali: Has my updated answer helped? $\endgroup$
    – csgillespie
    Nov 9, 2010 at 9:34
  • $\begingroup$ I am afraid my knowledge of particle filters is close to zero! How does this method of picking a particle with probability 1/M be then used with the new likelihood to walk over the posterior for the parameters? $\endgroup$
    – user28
    Nov 9, 2010 at 12:14
  • $\begingroup$ Suppose my prior on $\mu$ was N(0,1). A possible proposal would be to just select values from N(0,1) and accept/reject as necessary. In a particle representation, we could generate $M$ RNs from the N(0, 1) and select a particle at random. As M increases, the particle representation becomes as exact. $\endgroup$
    – csgillespie
    Nov 9, 2010 at 15:28
  • $\begingroup$ To be concrete, let: $f(\theta_c|-)$ be the current posterior for which I have MCMC samples. $L(y|\theta)$ be the likelihood corresponding to the new data point. Thus, the new posterior I wish to sample from is: $f(\theta_n |-) \propto f(\theta_c|-) L(y|\theta)$. Does your suggestion amount to implicitly setting the proposal density for the MH algorithm to be $f(\theta_c|-)$ and then use the particle filter approach as an approximation to $f(\theta_c|-)$? $\endgroup$
    – user28
    Nov 9, 2010 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.