3
$\begingroup$

The deviance statistic is defined as

$$D(\mathbf{y}, \hat{\mu}) = 2 \Big( \log p(\mathbf{y} | \hat{\theta}_s) - \log p(\mathbf{y} | \hat{\theta}_0) \Big),$$

where $\hat{\mu} = \mathbb{E}(Y|\hat{\theta}_0)$. I see the motivation in why we would define the deviance as a difference of log-likelihoods or just the logarithm of the likelihood-ratio, but why the factor 2? Why square the ratio? Does this introduce new information that we otherwise would not have insight to?


Edit: I have yet to find a satisfying answer, but could it be to do with having an approximate chi-squared distribution?

$\endgroup$
1
  • 1
    $\begingroup$ Hello, I figured this isn't worth a full response, but your edit is still up about not having a satisfying answer. I found this Likelihood-ratio test page from your Wilks' theorem page, which says that multiplying by −2 ensures that, in Wilks' theorem, the likelihood ratio converges to being chi-squared distributed if the null hypothesis is true. It gives the reference Silvey, S.D. (1970). Statistical Inference. London: Chapman & Hall. pp. 112–114. So that's a direct answer. Hope that helps, I enjoyed your post. $\endgroup$
    – Erik
    Commented Apr 21, 2023 at 23:05

1 Answer 1

2
$\begingroup$

I'm not certain that this is the intended reason from the progenitor of the deviance measure, but it seems likely that the reason for the factor of two is to ensure that the deviance corresponds to an appropriate scaled version of the sum-of-squares in the normal case. That is, in the case where you have a GLM using a normal distribution with variance $\sigma^2$ (i.e., standard regression) you get deviance values:

$$\begin{equation} \begin{aligned} D_{TOT} = 2(\hat{\ell}_{s} - \hat{\ell}_0) &= \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{TOT}, \\[6pt] D_{REG} = 2(\hat{\ell}_{p} - \hat{\ell}_0) &= \frac{1}{\sigma^2} \sum_{i=1}^n (\hat{y}_i - \bar{y})^2 = \frac{1}{\sigma^2} \cdot SS_{REG}, \\[6pt] D_{RES} = 2(\hat{\ell}_{s} - \hat{\ell}_{p}) &= \frac{1}{\sigma^2} \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \frac{1}{\sigma^2} \cdot SS_{RES}, \\[6pt] \end{aligned} \end{equation}$$

where the log-likelihoods under the null model, actual model, and saturated model, are given respectively by:

$$\begin{equation} \begin{aligned} \hat{\ell}_0 = \ln p(y|\hat{\theta}_0) \quad \quad \quad \hat{\ell}_p = \ln p(y|\hat{\theta}_p) \quad \quad \quad \hat{\ell}_s = \ln p(y|\hat{\theta}_s). \end{aligned} \end{equation}$$

If you go through the algebraic derivation of these results you will see that the factor of two in the deviance cancels out with the $\frac{1}{2}$ in the exponential for the normal density. Presumably the reason to have this factor is to define the deviance measures in a way that naturally extends the sums-of-squares in standard Gaussian regression.

$\endgroup$
2
  • $\begingroup$ Hi @Ben , thanks, I think I can accept that! Does the 2 contribute to the statistic following a ChiSquared distribution at all? Wilks' Theorem $\endgroup$
    – fool
    Commented Dec 8, 2018 at 7:48
  • 1
    $\begingroup$ @Kevin C: Yes. Without the factor of two it would have a scaled cih-squared distribution (multiplied by $\tfrac{1}{2}$). $\endgroup$
    – Ben
    Commented Dec 8, 2018 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.