McCullagh & Nelder, 2nd edition, p 91 claim that to make comparisons "fair", it's best to use a single estimate of overdispersion parameter, usually derived from the most complex model. I noticed that the same thing is done in this SAS example. Burnham and Anderson (2nd edition, p 68) are also adamant about using just the global model to estimate the dispersion parameter and then plugging it into smaller models.
Does anyone know what is meant by "fair" and why re-estimating a variance parameter is unfair? If (which is very likely) the most complex model is overfitted then its deviance is too small, meaning that the dispersion parameter is underestimated. Is it still a good idea to use the biased estimate for all models?
McCullagh & Nelder (p 127) admit that estimating the dispersion parameter is similar to estimating $\sigma^2$ in linear regression. Correspondingly, I recalled a similar reasoning in Neter et al (4th edition, p 342) where $C_p$ criterion for linear models is described. We are suggested that the largest model delivers "an unbiased estimator of $\sigma^2$" that we are supposed to use to assess the bias-variance tradeoff for smaller models.
For both linear and binomial setups, all the authors above agree that one must get the structural part of the model right before estimating the dispersion parameter. However, if we know what the correct structural part is, it is not clear why we should proceed with model selection instead of just using the global model.
Another contradiction is that, if we want to use AIC and similar criteria for linear model selection, then I have never heard of a recommendation to estimate $\sigma^2$ using the global model and then plug it into the smaller ones. Likewise, I haven't heard of such recommendation for the dispersion parameter in Negative Binomial regression.
Some comments below suggested that the problem is that in Binomial regression $\phi$ is not estimated via MLE, but in Negative Binomial regression it is. So, as long as there is a way to estimate dispersion via MLE, no one has a problem with allowing the dispersion parameter to vary across models, and vice-versa? For instance, if we use Williams variance function
$Var[Y_i] = m_i\pi_i(1-\pi_i)[1 + \phi(m_i-1)]$
where $\phi$ is estimated by IWLS, then we must use the same $\phi$ for all the models in the pool. However, if we apply Beta-binomial regression that can be estimated by MLE given that $\alpha_i + \beta_i = c$, then it's fine to allow $c$ to vary across the models. The problem with such reasoning is that Beta-binomial is a particular case of Williams variance function, where $\phi = 1/(c+1)$. It doesn't make sense to disallow $\phi$ to vary simply because it was not estimated by "pure" MLE. IWLS and MLE estimates of $\phi$ are probably close anyway, and may be even identical because in practice MLE is often implemented via IWLS (e.g. Negative Binomial regression in R).