I have a set of samples taken from a population distributed with a Gamma distribution, so \begin{equation} f_X(x)=\frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} \end{equation} I should estimate $\alpha$ and $\beta$, so knowing that \begin{equation} \alpha=\frac{E[X]^2}{\text{Var}[X]},\quad \beta=\frac{E[X]}{\text{Var}[X]} \end{equation} My idea is to compute $\hat \mu, \hat \sigma^2$ from the set as estimators of the mean and average of the population, then find $\hat \alpha,\hat \beta$ as follows \begin{equation} \hat \alpha=\frac{\hat \mu^2}{\hat \sigma^2},\quad \hat \beta=\frac{\hat \mu}{\sigma^2} \end{equation}

I'm trying to apply this logic to a task with a set of 10000 samples, but approximations seems to be not good enough. Is there any flaw in this reasoning?

EDIT: I've also tried the Newton-Raphson method to find alpha. This method leads to approximations very similar to the ones that I get from the method of moments.

EDIT 2: The logic is correct. Problem was a wrong understanding of the results of the implementation of this logic: in my task there was just half of the data needed to get all the estimators required. That is true for any possible method. Also approximation was fine.


closed as unclear what you're asking by Stephan Kolassa, Michael Chernick, user158565, COOLSerdash, whuber Apr 23 at 14:37

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 3
    $\begingroup$ You're using the method-of-moments estimators, afaik. In what way to you find them unsatisfying? Another possibility would be to use the maximum likelihood estimators. $\endgroup$ – COOLSerdash Apr 20 at 17:04
  • 4
    $\begingroup$ What @COOLSerdash says. The gamma can be parameterized in two different ways, are you sure you are not confusing something there? That could be a simple explanation for why your results may be bad. $\endgroup$ – Stephan Kolassa Apr 20 at 17:07
  • $\begingroup$ @COOLSerdash yes it is. I assume that for 10000 samples the method should work fine to have a good approximation. $\endgroup$ – ezy Apr 20 at 18:35
  • 1
    $\begingroup$ FWIW, your expression for $f$ isn't quite correct. It's difficult to determine what you mean by "each letter with a unique ... combination" and it's impossible to determine what you mean by "unsatisfying." Could you clarify your question? $\endgroup$ – whuber Apr 23 at 4:00
  • 1
    $\begingroup$ Without explanation of, or evidence for, "unsatisfying" results this question remains unclear. $\endgroup$ – Nick Cox Apr 24 at 8:16

As COOLSerdash says, what you are doing is method of moments estimation: you estimate parameters of a distribution so the estimated distribution's moments match the ones you observe (here, the first two moments, the mean and the variance). This is a completely valid estimation method.

Alternatively, you could estimate the parameters through maximum likelihood. Wikipedia has the details for the gamma distribution.

However, the differences between the two estimation methods are typically not very large, unless you have very degenerate distributions.

If your model gives you unsatisfactory results, this does not seem to be due to problems with your estimation. It may simply be that fitting gammas to your word frequencies does not separate different words cleanly enough. You may want to think about different methods. For instance, fitting gammas separately to each word's incidence loses all the context. You may want to try methods that model context, such as Markov chains or LSTM/RNNs.

  • $\begingroup$ "For instance, fitting gammas separately to each word's incidence loses all the context." Just to understand the meaning of context: the given encoded sentence has no meaning and is randomly generated so I wouldn't expect to get any context information. Said that, I can't see the Markov chains approach to help me finding the parameters for each letter. Also I confirm the following sentence "It may simply be that fitting gammas to your word frequencies does not separate different words cleanly enough.", in fact the parameters that I find for a letter are similar to ones of other letters. $\endgroup$ – ezy Apr 21 at 7:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.