I have a dataset (>10,000 samples total) that strongly appears to be geometrically distributed. For this dataset I have a way of partitioning that makes sense theoretically and I would like to know if my theory is backed by said dataset, i.e., the same question usually answered using ANOVA analysis in the continuous, normally distributed case. My question is: how is this done with geometrically distributed data?

My approach would be to fit a geometric distribution (using least squares) to the whole dataset, and then to each partition of the partitioned dataset. I would then compute the ratio of the sum of errors in the partitioned case to the squared error for the total distribution, i.e., $\frac{\sum_{part}\textrm{RMSE}_{part}^2}{\textrm{RMSE}_{total}^2}$, to get a test score, similar to an ANOVA.

This is where I am stuck, as I don't know which distribution to compare it to. Can somebody give me an ELI5 explanation of why it is stupid and wrong to solve my problem in this way, and then tell me how to actually do it?


The geometric distribution is a special case of the negative binomial distribution, so I would try a negative binomial glm (generalized linear model.) Some details here: When to use Poisson vs. geometric vs. negative binomial GLMs for count data?.

The generalization of ANOVA in glm's is called the analysis of deviance, but in some computer systems (for instance R) it is done with the same function called anova. You say My approach would be to fit a geometric distribution (using least squares) to the whole dataset ..., but this is not the way to do it. With glm's one uses maximum likelihood methods, as explained in the linked post above, and its references. Also relevant is Correct use of Negative Binomial with a Geometric distribution in a mixed model (glmmPQL)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.