1
$\begingroup$

In their book Generalized Linear Models, McCullagh & Nelder seem to imply that the sampling distribution of the deviance is generally not known:

page from the book This is strange, because the deviance of a model $m$ is defined as $$ D = -2 \bigl(\ell_m(\hat\theta_\text{ML}; x) - \ell_s(\hat\theta_\text{ML}; x) \bigr) $$ where $s$ denotes the so-called "saturated" model. In other words, the deviance is simply the test statistic from the likelihood-ratio test for two nested models, which, as far as I know, has a well-known (asymptotic) distribution. From the Wikipedia:

A convenient result, attributed to Samuel S. Wilks, says that as the sample size $n$ approaches $\infty$, the test statistic $-2 \log(\Lambda)$ for a nested model will be asymptotically $\chi^2$-distributed with degrees of freedom equal to the difference in dimensionality of $\Theta$ and $\Theta_0$.

So, do we know the distribution of $D$, yes or no?

$\endgroup$
  • 3
    $\begingroup$ Only asymptotically and only under the smaller model... $\endgroup$ – Xi'an Feb 18 '16 at 20:52
  • 1
    $\begingroup$ @Xi'an to me that seems like it's almost an answer; certainly it seems to cover the central point. $\endgroup$ – Glen_b Feb 18 '16 at 22:08
2
$\begingroup$

A better idea is that instead of thinking about the sampling distribution you use Posterior samples $\tilde{y}$ using MCMC P($\tilde{y}|y)$ = $ \int_\theta P(\tilde{y}|\theta)P(\theta|y)d\theta$ to generate a bunch of Deviance samples $D = -2*\sum_ilog(P(\tilde{y}))$. MCMC methods allow you to generate posterior samples. Since posterior samples encapsulate the uncertainty it is propagated to the deviance and that is what you really want when you make decisions. You use measures like WAIC and LOO which average deviances in very specific ways that are derived from solid theory based in Cross Validation.

A really good computational reference is McElreath's new book's Chapter 6 and for theory on this papers like this and this one that explain these concepts.

Of course posterior estimation are not easy or possible in many models. But for most Linear, Generalized Linear(especially these) models these methods are easily applicable(Basically anywhere where you have a maximum likelihood that's well defined)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.