4
$\begingroup$

I ask, because it is very hard to find information regarding the beta distribution and the bayesian inference, where the beta distri is NOT the prior.

My goal is to identify or to improve the two parameters $\alpha$ and $\beta$ of a fitted beta distribution from binned data with the pdf

$ f(x|\alpha,\beta)=\frac{1}{\text{B}(\alpha,\beta)} x^{\alpha-1}(1-x)^{\beta-1} $.

My first attempts to estimate the parameters is the method of moments, see http://en.wikipedia.org/wiki/Beta_distribution#Method_of_moments .

This works fine. But if the parameters are varying by time, what can I do? I thought of a mean/variance update and another use of the moments method. But can I update the mean and the variance each time a new measurement is coming, only with the old values of the mean and variance? And how can I weight the new data, so that my "offline" estimated parameters are more important? I couldn't find a solution so I thought maybe bayesian updating be the trick.

Assume, $\alpha$ and $\beta$ are the parameters $\theta$. Is there a way to estimate them? The problem is, my new data is only a "bin".

For example: I've got a variable $X_k$, where k is the discrete time. $X_{k+1}=f(X_k|\alpha,\beta)$ is a function of the beta pdf. $\alpha$ and $\beta$ are estimated from following measurements:

enter image description here

The parameters are estimated with moment matching. Then later I get new data. But only a count more at e.g. bin number 2. And then another measurement at bin 5 etc. After 100 measurements or so (or maybe at each measurement), I'd like to update the parameters. But I don't have the initial data anymore, since it need a lot of memory. Can that be done somehow?

Thank you very much!

$\endgroup$
  • $\begingroup$ Are you simply asking for online updating of ordinary parameter estimates (add an observation, update a parameter estimate), or are you actually asking for a Bayesian approach to this problem (updating posterior distributions of parameters, given some prior)? $\endgroup$ – Glen_b -Reinstate Monica Oct 4 '14 at 21:37
  • $\begingroup$ I think if it is possible to time weight (i.e. give the offline calculated parameters more weight and update only slowly) the online update of ordinary parameter estimation, then this would be enough. $\endgroup$ – Jan Oct 4 '14 at 21:47
2
$\begingroup$

Updating: Online updating (where it's feasible) attempts to update parameter estimates so they give the same answer as you would get if you did it offline, just with faster calculation via updating than complete recalculation.

Since you're using method of moments for the beta, you'd use online updating of sample mean and variance (or sample mean sample raw second moment, which is even easier), and simply recalculate parameter estimates from those.

Fast, stable algorithms exist for this already. For example, there's an algorithm described here to update mean and variance. There are other posts here describing the same, or very similar, algorithms.

$\ $

Weighting: Certainly it's possible to weight data by recency (exponentially weighted moving averages are especially common), but it's not quite clear from your comment what you want to achieve there. If you can clarify, I may be able to say more.

Exponentially weighted means are discussed (with several approaches to computation) in the online book by Hyndman and Athana­sopou­los here.

Weighted variances are in this Wikipedia article, which also describes exponentially decreasing weights. This can be incorporated into a weighted updating algorithm, as here, though because of the simple nature of exponential weights, the code can be simplified somewhat, which is described in the question here.

Updating for ML estimation:

The beta has $\overline{\log(X_i)}$ and $\overline{\log(1-X_i)}$ as sufficient statistics. [I emphasize the mean rather than the sum because if you're doing this online, presumably sample sizes may get very large indeed; you don't want issues with underflow, for example.]

ML estimators for the beta involve solving a pair of equations in digamma functions involving $\alpha$, $\beta$ ($\psi(\alpha)$,$\psi(\beta)$, and $\psi(\alpha+\beta)$) and the two sufficient statistics.

The digamma function has a number of nice properties that make obtaining good approximations simple and fast; solving for them is often quite rapid (many packages have built in digamma and trigamma functions, including R).

Updating the MLE's of $\alpha$ and $\beta$ would then involve updating the sufficient statistics (which are just means of transformed observations, so easily updated) and then re-solving for $\hat{\alpha}$ and $\hat{\beta}$. These equations will need to be solved iteratively, but because of the nice properties, the iterations can be started from a shift from the previous estimate that will be very close to the final estimate.

With some care in implementation, fairly accurate online MLEs for the beta should be quite feasible.

$\endgroup$
  • $\begingroup$ I'm not quite done yet, I'm digging up a link on exponential weighting for you now. $\endgroup$ – Glen_b -Reinstate Monica Oct 4 '14 at 23:21
  • $\begingroup$ Shit, I used too much time for editing my first comment, sorry :(. Thank you very much! What I mean with weighting is: I have 1000 datapoints, used to calculate the parameters offline. Then I go on the "road" and new measurements are taken. If I update the mean and the variance with an recursive formular there is nothing that says like: "the current mean was calculated by the first 1000 points. Trust it more". Or the other way round: Newer data should contain newer information, discard very old ones. $\endgroup$ – Jan Oct 4 '14 at 23:26
  • $\begingroup$ Thank you for your edit. Can I use the "usual" exponential ma, from signal processing? And I don't get it 100%. If there is a way to update the parameters recursive, maybe with an moving average filter, why should I use the bayesian way? Is it all about the a posteriori distribution? Because to get the parameter, I use the mean, correct? Thank you very much $\endgroup$ – Jan Oct 5 '14 at 0:13
  • $\begingroup$ I got interrupted during my previous edits. Now complete. I am not quite sure what you're asking. What's the '"usual" exponential ma, from signal processing', exactly? I'm not saying you should or should not use a Bayesian approach, I just wanted to be clear what you were asking about in your question. If you want to ask a question about the distinction between online updating and Bayesian updating, that might be a new question. $\endgroup$ – Glen_b -Reinstate Monica Oct 5 '14 at 1:26
  • $\begingroup$ When you say, "updating the mles involve updating the sufficient statistic", do you recommend to use the recursive version of the mean/variance formular? Would the MLE thing only an "update" to the method of moments? So I'd have to update the mean anyway somehow (i.e. with the exponential MA filter?)? $\endgroup$ – Jan Oct 6 '14 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.