Bayesian Inference: Feeding Posterior back in as Prior

Question

I've just started reading about Bayesian Inference, and one thing I've wondered about is if it's possible to feed the posterior in as a new prior for a new model, using the same data. Or would that somehow be redundant, as in it would just accept with itself?

I.e -

First model: posterior $\propto$ likelihood x prior

Second model: newposterior $\propto$ likelihood x posterior

EDIT: Maybe it would be helpful if we assume that the prior and likelihood are conjugate priors, which would yield the same distribution family for the posterior and prior.

Answer 1

This is effectively the same thing as duplicating the rows of your data - i.e. you artificially and inappropriately pretend that you had observed the same data you only saw once multiple times. If you do this $K$ times, you end up with $\text{"posterior"} \propto \text{likelihood}^K \text{prior}.$

Why is this a bad idea? Let's say I flip a coin 1 time and observe heads. I had a $\text{Beta}(50, 50)$ prior (prior median 0.500 with 95% credible interval from 0.40 to 0.60) for the proportion of times this particular coin lands heads-up. I.e. I a-priori think it's pretty likely that this is a reasonably fair coin, but it's not totally symmetrical so I think it's plausible that it could be alittle bit more likely to fall one one side or the other. If I analyze this data, I get a $\text{Beta}(51, 50)$ posterior (posterior median 0.505 with 95% credible interval 0.41 to 0.60) - barely any change, becuase 1 coin flip just tells me very little compared to how much I thought I knew before (which is equivalent to seeing 50 heads and 50 tails before). If I now do what you proposed 1000 times, I get a $\text{Beta}(1050, 50)$ "posterior" (with "posterior" median 0.955 with "credible" interval 0.94 to 0.97) - i.e. I am suddenly "completely convinced" that this coin comes up heads >90% of the time. Of course that does not make any sense when I've really only seen one single coin flip.

Answer 2

There is quite a Bayesian literature on "not using the data twice", starting from it being a probability non-sense [e.g., what happens to the dominating measure?] to biasing the inference towards over-fitting. There are however a few positive proponents:

1. Murray Aitkin defends this approach as a way to bypass paradoxes (Lindley's) and difficulties related with improper priors, especially in testing hypotheses. He calls this integrated likelihood and has a book and a Read Paper on the subject. The discussion at the end of the paper is illuminating on why this is incoherent, if ferocious at times. We also wrote a critical analysis of the book for Statistical Science.
2. If embracing the notion that feeding the prior with a power of the likelihood concentrates on the MLE, this may become a way to derive the MLE. I proposed this method in 1993 under the name of prior feedback, with an improved version (with Arnaud Doucet and Simon Godsill] under the acronym of SAME and it has been rediscovered since then under other names, like data cloning.

Answer 3

If the data being used to generate the likelihood is the same in both cases, then I'd say the answer is no - you can't use the posterior of the first calculation as the prior for the second.

You can think about this from the perspective of the definition of the prior and the likelihood. The prior is the probability of the model parameters not conditional on the experimental data being used to generate the likelihood.

If the data is the same in both cases then this is no longer the case.