I have implemented the normal inverse gamma distribution per section 3 of https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf in some code. However, I've noticed something that I didn't expect. The posterior differs depending on how I choose to update.
So, imagine I have data with n=100. If I update the posterior by taking my prior, updating with n=10, using that posterior as my prior, and then continuing that process until I've updated with all of the 100 data points, my variance (i.e. inverse-gamma portion of the distribution) is different than if I just do one update with the entire set of 100 points once.
Am I wrong in thinking that the posterior should be the same independent of how I choose to update? Am I missing something in how the posterior is computed?
Edit: adding the posterior function for the inverse-gamma:
$\sigma^{2}|x \sim Inv-Ga(\alpha + \frac{n}{2}, \beta + \frac{1}{2}\sum(x_{i}-\bar{x})^2 + \frac{nn_{0}}{2(n + n_{0})}(\bar{x} - \mu_{0})^2)$
Parameter updating: $NIG(\frac{\nu\mu_{0}+n\bar{x}}{\nu+n}, \nu+n,\alpha + \frac{n}{2}, \beta + \frac{1}{2}\sum(x_{i}-\bar{x})^2 + \frac{n\nu}{2(n + \nu)}(\bar{x} - \mu_{0})^2)$
Here's our Scala implementation in case anyone is interested: https://github.com/udemy/statistics/blob/master/src/main/scala/com/udemy/statistics/distribution/NormalInverseGamma.scala