How can posterior be persisted and reconstituted as future prior? Suppose I model a data generating process as a hierarchal model and have made some training observation from the process.   To learn about the process, with the observations I run the bayesian machinery to calculate posterior for the  stochastic parameters of the model.  Now I would like to be able to throw out the observations and just use the posterior I got in the learning process as prior for learning from the next set of observations.  
For cases where I don't have conjugate prior story, I am not sure how one can persist the state of posterior  (while throwing away all your observations) so that it can be used as prior for the next observation?
Any thoughts?
 A: You don't absolutely need a conjugate prior; it just makes your life considerably easier. For any probability distribution(s), it is true that
$$ P(\textrm{A } | \textrm{ B}) = \frac{P(\textrm{B } | \textrm{ A}) \cdot P(\textrm{A})}{P(\textrm{B})}$$
However, a conjugate prior helps in two ways:


*

*It can be messy--and difficult to do the updating for two arbitrary distributions. There often may not be a closed form solution, which means you'll have to fiddle around with potentially complicated and inexact numerical solutions.

*It simplifies the interpretation. Suppose you have a normal likelihood function and a normal prior. The resulting distribution is also a normal distribution. After several rounds of updating, you could then make well-supported claims about the mean and variance. This is often useful. On the other hand, if your end goal is to be able to use your fitted $P(A|B)$ to calculate $P(A'|B)$ for some new data $A'$, then maybe you would not care too much.
There is a handy Compendium of Conjugate Priors, compiled by Daniel Fink, that might help you find one that matches your likelihood function. 
