Skip to main content
Tweeted twitter.com/StackStats/status/1558694939269255169
deleted 30 characters in body
Source Link
Dave
  • 67.2k
  • 7
  • 105
  • 305

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a Poisson GLM.

However, I wonder if I might get a better fit by using a negative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Poisson model and the negative binomial model? That is:

$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent. Kjetil's answer here seems to indicate that such a comparison would not be meaningful for a Poisson vs a Gaussian (for example) likelihood, due to the discrete vs continuous dominating measures, but for Poisson vs negative binomial, both dominating measures are discrete.

(A similar setup would be if it is meaningful to compare a linear regression fitted by minimizing square loss (maximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE), which seems like an unfair comparison. At the same time, both dominating measures would be continuous.)

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a Poisson GLM.

However, I wonder if I might get a better fit by using a negative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Poisson model and the negative binomial model? That is:

$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent. Kjetil's answer here seems to indicate that such a comparison would not be meaningful for a Poisson vs a Gaussian (for example) likelihood, due to the discrete vs continuous dominating measures, but for Poisson vs negative binomial, both dominating measures are discrete.

(A similar setup would be if it is meaningful to compare a linear regression fitted by minimizing square loss (maximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE), which seems like an unfair comparison. At the same time, both dominating measures would be continuous.)

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC).

Let's say that I fit a Poisson GLM.

However, I wonder if I might get a better fit by using a negative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Poisson model and the negative binomial model? That is:

$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent. Kjetil's answer here seems to indicate that such a comparison would not be meaningful for a Poisson vs a Gaussian (for example) likelihood, due to the discrete vs continuous dominating measures, but for Poisson vs negative binomial, both dominating measures are discrete.

(A similar setup would be if it is meaningful to compare a linear regression fitted by minimizing square loss (maximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE), which seems like an unfair comparison. At the same time, both dominating measures would be continuous.)

deleted 194 characters in body
Source Link
Dave
  • 67.2k
  • 7
  • 105
  • 305

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a GaussianPoisson GLM, so the usual OLS linear regression.

However, I know that my $Y$ is always an integer, so I wonder if I might get a better fit by using an integer-only likelihood, and I fit a Poisson GLMnegative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the GaussianPoisson model and the Poissonnegative binomial model? That is:

$$ \mathbb E[Y\vert X] = \beta_0 +\beta_1X \implies AIC_1\text{, based on Gaussian likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on Poisson likelihood} $$$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent, and the. Kjetil's answer here also seems to indicate that it issuch a comparison would not be meaningful to compare $AIC_1$ andfor a Poisson vs a Gaussian $AIC_2$(for example) likelihood, butdue to the measure theory is a bit muchdiscrete vs continuous dominating measures, but for mePoisson vs negative binomial, both dominating measures are discrete.

(Then again, Kjetil's, "If one contemplates both continuous and discrete models, these two kinds of models cannotA similar setup would be compared with AIC, since they use different dominating measuresif it is meaningful to compare a linear regression fitted by minimizing square loss (Lebesgue measure, counting measuremaximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE)," which seems pretty definitivelike an unfair comparison. At the same time, both dominating measures would be continuous.)

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a Gaussian GLM, so the usual OLS linear regression.

However, I know that my $Y$ is always an integer, so I wonder if I might get a better fit by using an integer-only likelihood, and I fit a Poisson GLM.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Gaussian model and the Poisson model? That is:

$$ \mathbb E[Y\vert X] = \beta_0 +\beta_1X \implies AIC_1\text{, based on Gaussian likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on Poisson likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent, and the answer here also seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the measure theory is a bit much for me.

(Then again, Kjetil's, "If one contemplates both continuous and discrete models, these two kinds of models cannot be compared with AIC, since they use different dominating measures (Lebesgue measure, counting measure)," seems pretty definitive.)

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a Poisson GLM.

However, I wonder if I might get a better fit by using a negative binomial distribution.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Poisson model and the negative binomial model? That is:

$$ \log(\mathbb E[Y\vert X]) = \alpha_0 +\alpha_1X \implies AIC_1\text{, based on Poisson likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on negative binomial likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent. Kjetil's answer here seems to indicate that such a comparison would not be meaningful for a Poisson vs a Gaussian (for example) likelihood, due to the discrete vs continuous dominating measures, but for Poisson vs negative binomial, both dominating measures are discrete.

(A similar setup would be if it is meaningful to compare a linear regression fitted by minimizing square loss (maximum likelihood estimation for a Gaussian likelihood) and a linear regression fitted by minimizing absolute loss (maximum likelihood estimation for a Laplace likelihood). That seems like comparing MSE and MAE (or RMSE and MAE), which seems like an unfair comparison. At the same time, both dominating measures would be continuous.)

Post Undeleted by Dave
Post Deleted by Dave
Post Undeleted by Dave
Post Deleted by Dave
Source Link
Dave
  • 67.2k
  • 7
  • 105
  • 305

Compare GLM AICs with different likelihoods?

If I have a generalized linear model (GLM) with a particular likelihood, and I have another GLM of the same data (say nested within the first model), I can compare the model performance using Akaike information criterion (AIC) and say which model is better.

Let's say that I fit a Gaussian GLM, so the usual OLS linear regression.

However, I know that my $Y$ is always an integer, so I wonder if I might get a better fit by using an integer-only likelihood, and I fit a Poisson GLM.

Assuming the same features, what, if any, meaning is there to the difference in AIC values between the Gaussian model and the Poisson model? That is:

$$ \mathbb E[Y\vert X] = \beta_0 +\beta_1X \implies AIC_1\text{, based on Gaussian likelihood}\\ \log(\mathbb E[Y\vert X]) = \beta_0 +\beta_1X \implies AIC_2\text{, based on Poisson likelihood} $$

Is there any meaning to comparing $AIC_1$ and $AIC_2?$ The answer here seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the comments express some dissent, and the answer here also seems to indicate that it is not meaningful to compare $AIC_1$ and $AIC_2$, but the measure theory is a bit much for me.

(Then again, Kjetil's, "If one contemplates both continuous and discrete models, these two kinds of models cannot be compared with AIC, since they use different dominating measures (Lebesgue measure, counting measure)," seems pretty definitive.)