# Difference between sequential and one-batch Bayesian update

I learnt that sequential Bayesian update and one batch (all at once)update will give the same result if the observations are i.i.d. I tried to test this theory using my model which contains 4 parameters (2 of them are highly correlated). Here is how I implement it:

1. I divide 500 observations into 10 sets of 50data for sequential update
2. Define the prior distributions of the parameters. (assume all follows N(ai, bi))
3. observe one set of 50 data, the conditional posterior samples of each parameter are obtained by Gibbs sampling
4. obtain the mean and std from the posterior samples and fit into normal distribution as new priors for the next set of data
5. repeat steps 2-4 10 times

I found that the results between sequential and batch differ a lot. Can anyone please suggest to me what causes the difference?

If you reduce things to a super simple example, your approach should work though. E.g. it should work for the case where you only unknown parameter is the mean of a normal distribution (with known SD) as you can see here. In that simple case, you can do this both analytically using conjugate updating (see first row of this table), or numerically using conjugate updating (the RBesT R package is a nice tool for letting the computer do this for you), or in the way you described. However, once you assume the standard deviation is unknown, you already end up with a normal distribution being wrong even as the marginal posterior distribution for the mean (too short-tailed; additionally, of course you need a joint posterior for both parameters).
• Yes, you could do that, but note: a multivariate normal cannot always describe your posterior well. A workaround is to use a mixture of multivariate normal distributions. E.g. in this article I used the mclust R package to do this (code can be found here). However, note that this very quickly becomes challenging, if you have too many parameters. The example was just $p=2$ hyper-parameters (not sure above which $p$ things get difficult). Sep 20, 2022 at 11:33