The deviance statistic is defined as

$$D(\mathbf{y}, \hat{\mu}) = 2 \Big( \log p(\mathbf{y} | \hat{\theta}_s) - \log p(\mathbf{y} | \hat{\theta}_0) \Big),$$

where $\hat{\mu} = \mathbb{E}(Y|\hat{\theta}_0)$. I see the motivation in why we would define the deviance as a difference of log-likelihoods or just the logarithm of the likelihood-ratio, but why the factor 2? Why square the ratio? Does this introduce new information that we otherwise would not have insight to?

Edit: I have yet to find a satisfying answer, but could it be to do with having an approximate chi-squared distribution?


I'm not certain that this is the intended reason from the progenitor of the deviance measure, but it seems likely that the reason for the factor of two is to ensure that the deviance corresponds to an appropriate scaled version of the sum-of-squares in the normal case. That is, in the case where you have a GLM using a normal distribution with variance $\sigma^2$ (i.e., standard regression) you get deviance values:

$$\begin{equation} \begin{aligned} D_{TOT} = 2(\hat{\ell}_{s} - \hat{\ell}_0) &= \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{TOT}, \\[6pt] D_{REG} = 2(\hat{\ell}_{p} - \hat{\ell}_0) &= \frac{1}{\sigma^2} \sum_{i=1}^n (\hat{y}_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{REG}, \\[6pt] D_{RES} = 2(\hat{\ell}_{s} - \hat{\ell}_{p}) &= \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \frac{1}{\sigma^2} \cdot SS_{RES}, \\[6pt] \end{aligned} \end{equation}$$

where the log-likelihoods under the null model, actual model, and saturated model, are given respectively by:

$$\begin{equation} \begin{aligned} \hat{\ell}_0 = \ln p(y|\hat{\theta}_0) \quad \quad \quad \hat{\ell}_p = \ln p(y|\hat{\theta}_p) \quad \quad \quad \hat{\ell}_s = \ln p(y|\hat{\theta}_s). \end{aligned} \end{equation}$$

If you go through the algebraic derivation of these results you will see that the factor of two in the deviance cancels out with the $\frac{1}{2}$ in the exponential for the normal density. Presumably the reason to have this factor is to define the deviance measures in a way that naturally extends the sums-of-squares in standard Gaussian regression.

  • $\begingroup$ Hi @Ben , thanks, I think I can accept that! Does the 2 contribute to the statistic following a ChiSquared distribution at all? Wilks' Theorem $\endgroup$
    – user
    Dec 8 '18 at 7:48
  • 1
    $\begingroup$ @Kevin C: Yes. Without the factor of two it would have a scaled cih-squared distribution (multiplied by $\tfrac{1}{2}$). $\endgroup$
    – Ben
    Dec 8 '18 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.