# Is it possible to apply Bayes Theorem with only samples from the prior?

I was just wondering-- is it possible/practical to apply bayes' theorem without an analytical expression for the prior, only samples?

For example, say you have sufficient draws from a posterior distribution from a previous experiment via MCMC methods, and you'd like to use that posterior as the prior for a new one. You have an analytical expression for the likelihood as before, but now only samples from the (new) prior. How would you proceed?

The short answer is yes. Have a look at sequential MCMC/ particle filters.

Essentially, your prior consists of a bunch of particles ($M$). So to sample from your prior, just select a particle with probability $1/M$. Since each particle has equal probability of being chosen, this term disappears in the M-H ratio.

A big problem with particle filters is particle degeneracy. This happens because you are trying to represent a continuous distribution with discrete particles - there's no such thing as a free lunch!

Clarification for Srikant Vadali

The question as I read it is: I have output, i.e. posterior from a MCMC scheme. I want to use this posterior as a prior for a new data set.

This (probably) means that you have a discrete representation of a continuous distribution, i.e. a particle representation. So rather than doing a random walk on a continous distribution (say), you need to pick values from your prior, i.e. you pick a particle.

Toni et al., use this idea with ABC.

• I am not sure I follow the suggestion properly. How does the idea of a particle filter help here?
– user28
Nov 8, 2010 at 19:50
• @Srikant-vadali: Has my updated answer helped? Nov 9, 2010 at 9:34
• I am afraid my knowledge of particle filters is close to zero! How does this method of picking a particle with probability 1/M be then used with the new likelihood to walk over the posterior for the parameters?
– user28
Nov 9, 2010 at 12:14
• Suppose my prior on $\mu$ was N(0,1). A possible proposal would be to just select values from N(0,1) and accept/reject as necessary. In a particle representation, we could generate $M$ RNs from the N(0, 1) and select a particle at random. As M increases, the particle representation becomes as exact. Nov 9, 2010 at 15:28
• To be concrete, let: $f(\theta_c|-)$ be the current posterior for which I have MCMC samples. $L(y|\theta)$ be the likelihood corresponding to the new data point. Thus, the new posterior I wish to sample from is: $f(\theta_n |-) \propto f(\theta_c|-) L(y|\theta)$. Does your suggestion amount to implicitly setting the proposal density for the MH algorithm to be $f(\theta_c|-)$ and then use the particle filter approach as an approximation to $f(\theta_c|-)$?
– user28
Nov 9, 2010 at 15:40