In a part of my research, I am fitting probability distribution models for the count data(and binomial data) using Poisson, Binomial, Negative Binomial and Beta Binomial models. I have few data sets in which the observed frequency and hence the expected modeled frequencies are too small(less than 5).. I want to compare the goodness of fits using a proper method. I know, Pearson's Chi-Square test will not be applicable for my purposes as it has some limitations on expected counts.

Here is a similar situation which I also deal with.. enter image description here

I am looking for an alternative Goodness of fit test measure which can be applied to this kind of situations.

Any suggestion is greatly appreciated.. Thank you.

  • 2
    $\begingroup$ If you estimate this by Maximum Likelihood, can't you just compare loglikelihoods? Whichever distribution family has higher LL wins the tournament. $\endgroup$ – Nameless Mar 29 '13 at 19:08
  • 3
    $\begingroup$ the 'expected count greater than 5' rule is about 50 years old. More recent studies suggest various relaxations of that under particular conditions (if the expectations are close to equal or only a small proportion of cells have low expected counts). You can always simulate null distributions of Pearson- or G-type statistics, or try an exact test if you have software to do that. $\endgroup$ – Glen_b Mar 30 '13 at 1:09
  • $\begingroup$ Thank you @Nameless , I am also considering about comparing those models via log-likelihoods. Indeed, I have already found Negative LL and chosen the best model which gives minimum NLL. Once again, thank you. $\endgroup$ – Arun Mar 30 '13 at 6:12
  • $\begingroup$ @Glen_b Your comment seems interesting. I have read few papers which are related to your comment. So, If some other conditions are satisfied(number of categories, number of observations), it is not necessary to restrict with this (<5) assumption. However, from the above given table, the last three frequency distributions do not even satisfy other relaxations. Thus, as Nameless suggested, LL comparison if one of the possible ways and, as you are suggesting, ///simulating null distributions of Pearson- or G-type statistics// is the other option. $\endgroup$ – Arun Mar 30 '13 at 6:18
  • 1
    $\begingroup$ I think an answer to that would be extensive enough to merit a new question. $\endgroup$ – Glen_b Mar 30 '13 at 7:21

I think you are asking for the "Multinomial Exact Test", which can exactly compute the p-value for whether a multinomial random variable (which takes any of a certain set of values) follows a certain distribution.

| cite | improve this answer | |

You can use Fisher's Exact test (also known as the multinomial test described above). https://en.wikipedia.org/wiki/Fisher%27s_exact_test

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.