Compare GLM AICs with different likelihoods?

Question

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC).

Let's say that I fit a Poisson GLM.

However, I wonder if I might get a better fit by using a negative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Poisson model and the negative binomial model? That is:

$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent. Kjetil's answer here seems to indicate that such a comparison would not be meaningful for a Poisson vs a Gaussian (for example) likelihood, due to the discrete vs continuous dominating measures, but for Poisson vs negative binomial, both dominating measures are discrete.

(A similar setup would be if it is meaningful to compare a linear regression fitted by minimizing square loss (maximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE), which seems like an unfair comparison. At the same time, both dominating measures would be continuous.)

Answer 1

The Aikake information criterion (AIC) is derived by minimizing the Kullback–Leibler (KL) divergence between the data-generating distribution $g(y)$ and an approximating model $f_\theta(y)$ with parameters $\theta$. Asymptotically, KL$(g \| f_\theta)$ is minimized by the maximum likelihood estimator $\hat{\theta}_n$.

So we can use AIC to compare two generalized linear models (GLMs) — even if the likelihood functions are different — if we use (a) maximum likelihood and (b) the same data to estimate the model parameters $\theta_1$ and $\theta_2$. The comparison will tell us which model, $f_{\theta_1}$ or $f_{\theta_2}$, is the "closer approximation" for the truth $g$.

The theory behind AIC doesn't make assumptions about the form of the models $f_{\theta_1}$ and $f_{\theta_2}$, except the usual "mild regularity conditions". "Same data" means the same observations (data points) and the same outcome variable for each observation. Basically, the same data-generating distribution $g(y)$. That's why we can't compare a model for $y$ and one for a transformation such as $\log(y)$; a model for a continuous $y$ and one for a discretized version of $y$; two models with different supports for $y$, etc.

Answer 2

If we dissect AIC formula by Akaike (1974), $$ AIC=-2\ln(L)+2k, $$ where $k$ is the number of parameters, the two differences between $AIC_1$ and $AIC_2$ are the value of $k$ (negative binomial likelihood will have an extra parameter) and the $\ln(L)$ values. Both are discrete regression models and they are often used to compare two or more models. However, one may argue if the difference in $AIC_1$ and $AIC_2$ are significant enough to conclude one model is better than the other. In this case, one may employ a likelihood ratio test and decide which model is the best.

See Likelihood Ratio Test for Poisson vs NB GLM