Recursively updating the parameter of a Beta function in a bayesian way?

Question

I ask, because it is very hard to find information regarding the beta distribution and the bayesian inference, where the beta distri is NOT the prior.

My goal is to identify or to improve the two parameters $\alpha$ and $\beta$ of a fitted beta distribution from binned data with the pdf

$ f(x|\alpha,\beta)=\frac{1}{\text{B}(\alpha,\beta)} x^{\alpha-1}(1-x)^{\beta-1} $.

My first attempts to estimate the parameters is the method of moments, see http://en.wikipedia.org/wiki/Beta_distribution#Method_of_moments .

This works fine. But if the parameters are varying by time, what can I do? I thought of a mean/variance update and another use of the moments method. But can I update the mean and the variance each time a new measurement is coming, only with the old values of the mean and variance? And how can I weight the new data, so that my "offline" estimated parameters are more important? I couldn't find a solution so I thought maybe bayesian updating be the trick.

Assume, $\alpha$ and $\beta$ are the parameters $\theta$. Is there a way to estimate them? The problem is, my new data is only a "bin".

For example: I've got a variable $X_k$, where k is the discrete time. $X_{k+1}=f(X_k|\alpha,\beta)$ is a function of the beta pdf. $\alpha$ and $\beta$ are estimated from following measurements:

The parameters are estimated with moment matching. Then later I get new data. But only a count more at e.g. bin number 2. And then another measurement at bin 5 etc. After 100 measurements or so (or maybe at each measurement), I'd like to update the parameters. But I don't have the initial data anymore, since it need a lot of memory. Can that be done somehow?

Thank you very much!

Answer 1

Updating: Online updating (where it's feasible) attempts to update parameter estimates so they give the same answer as you would get if you did it offline, just with faster calculation via updating than complete recalculation.

Since you're using method of moments for the beta, you'd use online updating of sample mean and variance (or sample mean sample raw second moment, which is even easier), and simply recalculate parameter estimates from those.

Fast, stable algorithms exist for this already. For example, there's an algorithm described here to update mean and variance. There are other posts here describing the same, or very similar, algorithms.

$\ $

Weighting: Certainly it's possible to weight data by recency (exponentially weighted moving averages are especially common), but it's not quite clear from your comment what you want to achieve there. If you can clarify, I may be able to say more.

Exponentially weighted means are discussed (with several approaches to computation) in the online book by Hyndman and Athana­sopou­los here.

Weighted variances are in this Wikipedia article, which also describes exponentially decreasing weights. This can be incorporated into a weighted updating algorithm, as here, though because of the simple nature of exponential weights, the code can be simplified somewhat, which is described in the question here.

Updating for ML estimation:

The beta has $\overline{\log(X_i)}$ and $\overline{\log(1-X_i)}$ as sufficient statistics. [I emphasize the mean rather than the sum because if you're doing this online, presumably sample sizes may get very large indeed; you don't want issues with underflow, for example.]

ML estimators for the beta involve solving a pair of equations in digamma functions involving $\alpha$, $\beta$ ($\psi(\alpha)$,$\psi(\beta)$, and $\psi(\alpha+\beta)$) and the two sufficient statistics.

The digamma function has a number of nice properties that make obtaining good approximations simple and fast; solving for them is often quite rapid (many packages have built in digamma and trigamma functions, including R).

Updating the MLE's of $\alpha$ and $\beta$ would then involve updating the sufficient statistics (which are just means of transformed observations, so easily updated) and then re-solving for $\hat{\alpha}$ and $\hat{\beta}$. These equations will need to be solved iteratively, but because of the nice properties, the iterations can be started from a shift from the previous estimate that will be very close to the final estimate.

With some care in implementation, fairly accurate online MLEs for the beta should be quite feasible.