# Beta binomial Bayesian updating over many iterations

I'm using a beta binomial updating model for a piece of code that I am writing. The software is real time updating - meaning that data is continually being gathered and after N data points are gathered, the bayesian model is updated using the N data points.

Under this logic, I am using the posterior output as my prior for the next iteration. My problem is that over billions/trillions/maybe more of iterations, the bayesian beta parameters (alpha and beta) will grow very large. I am worried that eventually the parameters will become so large that they will cause an integer overflow in memory.

So my question is twofold -

1. Is it reasonable to be worried about this integer overflow. I understand that $2^{32}$ is an extremely large number, but I'm building this software for an internet service that will be running 24/7, 365 days a year and I don't want it to crash. For example if I was updating it with 1,000,000 data points a day then the model would only last ~4000 days before an integer overflow.

2. Is it possible to transform a Beta(x,y) r.v., where x and y are extremely large, to a Beta(x*,y*) r.v. where x* and y* are relatively smaller? The transformed Beta doesn't have to be exact, just similar.

-

1) You could scale it down, so $\alpha,\beta\mapsto \alpha/N, \beta/N$. This would indeed allow you to continue. What this would do, however, is to make older data carry less weight (if $N$ is two, it would be carrying half as much weight). This might even be a feature, if you would rather trust newer data.

Compare for example $\alpha=\beta=20$ and $\alpha=\beta=10$ here. What you are doing when dividing by $N$ is multiplying the variance of the distribution by $N$ (almost!) while leaving the expected value unaffected.

2) You could stop right there. With 1 million data points, you distribution will essentially be a point. If you are having troubles with your model, despite 1000000 data points, you don't need more data, you need a better model.

In short, overflow shouldn't be a problem with a binomial-beta setup, because long before you reach overflow, you will have insanely small confidence intervals.

-
what if the data isn't always coming from the same distribution though (or the data distribution changes over time)? –  Michael Jul 27 '11 at 13:47
(I presume a sort of ad hoc engineering approach here) Then I would suggest dividing by an $N$ every week (or day, hour, etc..). I.e., (1) above. This will discount observations last week by $N$, observations from the week before that by $N^2$ and so on. What you are in effect doing is a weighted average where you give more weight to more recent observations. –  Har Jul 27 '11 at 14:07