Two GLMs of the same distribution are fit to the same set of data, using Maximum Likelihood Estimation (MLE) to fit the regression parameters $\beta$. The scale/dispersion parameter for each model is then separately estimated (using the same estimation, but not necessarily MLE).

Is it appropriate to calculate the AIC for each model, using the different scale parameters for each model, and use the difference to choose between them?

My motivation for this question is that for nested models $M_1 \subset M_2$, the differences in estimated scale parameters can make the calculated log likelihood of $M_1$ larger than $M_2$. My intuition for AIC is that the log likelihood measures how well we fit the data, and this is penalized by model complexity measured as 2 times the number of parameters. But that intuition appears to be wrong in this case, since the log likelihood of the more complicated model decreased all by itself, owing to the change in scale parameter.

As an aside (and potentially an answer), I assume the situation I mention could not happen if the scale parameter were also fit with MLE. Is it that such comparisons are valid only when the scale parameter is fit with MLE?

  • $\begingroup$ The scale is part of the likelihood and usually fitted with the model. Maybe you could give a bit more context - what kind of models are you estimating here, and why do you fix the scale by hand? $\endgroup$ – Florian Hartig Aug 11 '16 at 15:30
  • $\begingroup$ This paper, for example, gives three alternative methods of calculating the scale parameter and states that the calculation of the regression parameters is not affected by choice of the dispersion parameter. I am still going to let software calculate the parameter for me, I just want to be careful in how the software calculates this. $\endgroup$ – Jonny Lomond Aug 11 '16 at 15:53
  • $\begingroup$ Well, depends on the software I guess. That regression parameters are not affected (if that is true) does not say anything about how dispersion is estimated. Anyway, I just stumbled across this, maybe of interest stats.stackexchange.com/questions/226152/… $\endgroup$ – Florian Hartig Aug 11 '16 at 15:59
  • $\begingroup$ Thank you very much, I appreciate that reference. These notes also suggest that the scale parameter of two nested models should be estimated by that of the larger model, and uses this to derive the F-test statistic for linear models. (On page 7 of the notes) $\endgroup$ – Jonny Lomond Aug 11 '16 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.