F-test for nested GLM

Question

Assume we are given two nested GLM models $M_0 \subset M_1$ with $q$ and $p$ parameters respectively. We also know that dispersion parameter in both models is estimated as the same value, denote it $\widehat{\phi} \neq 1$. My goal is to test the significance of $p-q$ additional parameters in bigger model. In this case, the appropriate test would be F-test (see e.g. Section 4.7.6.2 here) with $$ F = \frac{D_0-D_1}{\widehat{\phi} (p-q)}, $$ where $D_0, D_1$ are deviances of $M_0$ and $M_1$. In the very same book the deviance is defined as $$ D = 2 \phi(L_\textrm{full} - L), $$ where $L_\textrm{full}$ and $L$ are log-likelihoods of full (saturated) model and considered model. By naive way, one can estimate the deviance by substituting $\widehat{\phi}$ for $\phi$ and hence in our case $$ F = \frac{2 \widehat{\phi} (L_\textrm{full} - L_0) - 2 \widehat{\phi}(L_\textrm{full} - L_1)}{\widehat{\phi} (p-q)} = \frac{2(L_1-L_0)}{p-q}. $$ On the other hand, I stumbled upon an exercise where $L_0, L_1$ and $\widehat{\phi}$ were given and the F-ratio was calculated as $$ F = \frac{2(L_1-L_0)}{\widehat{\phi}(p-q)}, $$ which made me think that scaled deviances ($D^* = D/\phi$) were used here instead of regular deviances.

Which way of computing this statistic is correct?

There are plenty of different sources I looked for it (e.g. here or here) but it seems to me that in each one something different is said about it, and I can’t reconcile all the versions. I would appreciate any help!

Answer 1

I strongly suspect that your first equation (unscaled deviances in the denominator) is correct, partly because it makes more sense in terms of units (e.g. in the version in your last displayed equation, we would effectively be dividing by the dispersion twice), and because Venables and Ripley *Modern Applied Statistics in S" (Springer, 2003), a classic and reliable reference, agrees with your first equation (p. 187):

---

Let $M_0 \subset M$ be a submodel with $q \lt p$ regression parameters and consider testing $M_0$ within $M$. If $\varphi$ is known, by the usual likelihood ratio argument under $M_0$ we have a test given by

$$ \frac{D_{M_0} -D_{M}}{\varphi} \overset{\cdot}{\underset{\cdot}{\sim}} \chi^2_{p-q} $$

where $\overset{\cdot}{\underset{\cdot}{\sim}}$ denotes “is approximately distributed as.” The distribution is exact only in the Gaussian family with identity link. If $\varphi$ is not known, by analogy with the Gaussian case it is customary to use the approximate result

$$ \frac{D_{M_0} -D_{M}}{\hat \varphi (p-q)} \overset{\cdot}{\underset{\cdot}{\sim}} F_{p-q, n-p} $$

although this must be used with some caution in non-Gaussian cases.

---

(as usual, it's not exactly clear what "some caution" means, exactly, except that one should be aware that the approximation might not be good ...)