Beta vs beta-binomial why beta has higher AIC

Question

I am working with proportion data (very limited ~20 data points) for a response variable (RV), i.e. proportion of mature females out of total number of females sampled. The maturity is assessed by 6 distinct maturity stages. I started out with a binomial distribution GLM (M3), which was overdispersed, I calculated the dispersion statistic to be ~115. Subsequently, I tried a beta distribution (M4), the model validation here suggested it was an okay fit (as okay as it can be with 20 dps). However, upon reading this paper I realized that the beta distribution is not strictly a good fit for my RV, because the proportion is derived from discrete data, as opposed to continuous. The paper pointed me to the beta-binomial distribution (M5), which I had no issue running. After that, out of curiosity I ran AIC() on all three models to compare them. This was the result:

AIC(M3, M4, M5)
   df        AIC
M3  2 2335.20106
M4  3  -14.56873
M5  3  265.97071

My question is, why is the beta model (M4)seemingly performing so much better than the beta-binomial (M5) according to AIC? Is this due to overfitting? Or should I not have done the beta model in the first place because its wrong for my type of proportion data, and just ignore AIC?

Answer 1

If you are modeling the proportion directly (i.e. you are measuring $\frac{\text{count of mature females}}{\text{count of all females}})$ then the beta distribution will have a much better AIC than the other options.

The reason for this is that AIC is a penalized log likelihood, where the likelihood is the product of probabilities of the observations under the model. The key here is the the beta distribution is a probability density function for a proportion, whereas the binomial and beta-binomial are probability mass functions for a count out of some total count. The beta distribution gives probability densities for real numbers in the interval $[0, 1]$, whereas the binomial and beta-binomial give probabilities masses to positive integers in the set $\{0, 1, 2, ..., n_i\}$ (where $n_i$ is the total count for the $i$th observation).

Aside from the fact that probability masses and densities are on different scales (masses must be in $[0, 1]$, but densities can be in $[0, \infty)$), if you were to model the proportion directly the beta is the only one of the three choices that would be able to give valid likelihoods (a count can't be fractional). Thus the beta would naturally have a better AIC.

Despite this, if you have the actual count data and not just the proportion, you should use it. The probability parameter of a binomial distribution, and the shape parameters of a beta-binomial distribution can give you estimates for the proportions using the count data. You will likely get much better estimates using the count data, as you will not throw away information (e.g. the total count).