69
$\begingroup$

How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also?

Null deviance: 1146.1 on 1077 degrees of freedom Residual deviance: 4589.4 on 1099 degrees of freedom AIC: 11089

Improve this question
$\endgroup$

3 Answers 3

Reset to default
108
$\begingroup$

Let LL = loglikelihood

Here is a quick summary of what you see from the summary(glm.fit) output,

Null Deviance = 2(LL(Saturated Model) - LL(Null Model)) on df = df_Sat - df_Null

Residual Deviance = 2(LL(Saturated Model) - LL(Proposed Model)) df = df_Sat - df_Proposed

The Saturated Model is a model that assumes each data point has its own parameters (which means you have n parameters to estimate.)

The Null Model assumes the exact "opposite", in that is assumes one parameter for all of the data points, which means you only estimate 1 parameter.

The Proposed Model assumes you can explain your data points with p parameters + an intercept term, so you have p+1 parameters.

If your Null Deviance is really small, it means that the Null Model explains the data pretty well. Likewise with your Residual Deviance.

What does really small mean? If your model is "good" then your Deviance is approx Chi^2 with (df_sat - df_model) degrees of freedom.

If you want to compare you Null model with your Proposed model, then you can look at

(Null Deviance - Residual Deviance) approx Chi^2 with df Proposed - df Null = (n-(p+1))-(n-1)=p

Are the results you gave directly from R? They seem a little bit odd, because generally you should see that the degrees of freedom reported on the Null are always higher than the degrees of freedom reported on the Residual. That is because again:

Null_Deviance_df = Saturated_df - Null_df = n-1

Residual_Deviance_df = Saturated_df - Proposed_df = n-(p+1)

Improve this answer
$\endgroup$
9
  • $\begingroup$ Yes, that's a very useful write-up @TeresaStat, thanks. How robust is this? Do the definitions change if you're talking about a multinomial model instead of a GLM? $\endgroup$
    – Hack-R
    Oct 31, 2014 at 17:00
  • $\begingroup$ @Teresa: Yes, these results are from R. Why would this happen? Any problem with the model here? $\endgroup$
    – Anjali
    Dec 20, 2014 at 9:44
  • $\begingroup$ @Hack-R: sorry for such a late response, I'm new to Stackexchange. For multinomial models you don't use the glm function in R and the output is different. You will need to look at either a proportional odds model or ordinal regression, the mlogit function. It is worth it to do a bit of reading on multinomial glms, they have slighty different assumptions. If I can get to it during the break, I'll update this with some more information. $\endgroup$
    – TeresaStat
    Dec 22, 2014 at 1:07
  • $\begingroup$ @Anjali, I'm not quite sure why you would get results like that in R. Its hard to know without seeing your data/results. In general, I don't see why the residual degrees of freedom would be higher than the null df. How many parameters were you estimating? $\endgroup$
    – TeresaStat
    Dec 22, 2014 at 1:09
  • 2
    $\begingroup$ @user4050 The goal of modeling in general can be seen as using the smallest number of parameters to explain the most about your response. To figure out how many parameters to use you need to look at the benefit of adding one more parameter. If an extra parameter explains a lot (produces high deviance) from your smaller model, then you need the extra parameter. In order to quantify what a lot is you need statistical theory. The theory tell us that the deviance is chi squared with degrees of freedom equal to the difference of parameters between your two models. Is it any clearer? $\endgroup$
    – TeresaStat
    Jan 28, 2015 at 23:34
20
$\begingroup$

The null deviance shows how well the response is predicted by the model with nothing but an intercept.

The residual deviance shows how well the response is predicted by the model when the predictors are included. From your example, it can be seen that the deviance goes up by 3443.3 when 22 predictor variables are added (note: degrees of freedom = no. of observations – no. of predictors) . This increase in deviance is evidence of a significant lack of fit.

We can also use the residual deviance to test whether the null hypothesis is true (i.e. Logistic regression model provides an adequate fit for the data). This is possible because the deviance is given by the chi-squared value at a certain degrees of freedom. In order to test for significance, we can find out associated p-values using the below formula in R:

p-value = 1 - pchisq(deviance, degrees of freedom)

Using the above values of residual deviance and DF, you get a p-value of approximately zero showing that there is a significant lack of evidence to support the null hypothesis. 

> 1 - pchisq(4589.4, 1099)
[1] 0
Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ How do you know what the cut off is for good/bad fit based on the deviance and number of predictor variables (without the pchisq)? Is it just if Residual Deviance > NULL Deviance or is there some range/ratio? $\endgroup$
    – Hack-R
    Aug 20, 2014 at 20:42
  • 3
    $\begingroup$ Your answer isn't wrong, but is subject to misunderstanding. In fact, it has been misunderstood (cf here). In light of that, can you clarify the differences that are implicit in your code? $\endgroup$
    – gung - Reinstate Monica
    Mar 10, 2015 at 18:06
  • $\begingroup$ It is often claimed that the residual deviance follows a chisq distribution when the fitted model contains all relevant predictors but there is no published theory to support this result and it actually true only for a very limited class of glms. Basically the result is only true for Poisson glms where all the counts are not tool small (all >= 3 say) and for binomial regression where all the proportions $y/n$ are well separated from 0 or 1. For other glms the deviance is either not chisq distributed or needs to be scaled by the dispersion parameter. $\endgroup$
    – Gordon Smyth
    Mar 17, 2021 at 4:29
  • $\begingroup$ ... Pete Dunn and I discuss this at length in our 2018 book on glms and give a theorem for the residual deviance using the saddle point approximation. $\endgroup$
    – Gordon Smyth
    Mar 17, 2021 at 4:34
  • $\begingroup$ statsgeek have shown, how this method of obtaining pvalue for the goodness of fit (for Poisson models) is misleading: thestatsgeek.com/2014/04/26/… $\endgroup$
    – Pawel
    Sep 14, 2021 at 22:38
3
$\begingroup$

While both answers given here are correct (and really useful resources), from page 432 of Introduction to Linear Regression Analysis (Montgomery, Peck, Vining, 5E), a general rule of thumb is given as if $$ \frac{D}{n-p} >> 1, $$ where $p$ is the number of regressors, $n$ is the number of observations and $D$ is the residual deviance, then the fit can be considered inadequate.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This rule of thumb is often quoted but is only useful for a very limited class of glms. Basically it only works for Poisson glms where all the counts are not tool small (all >= 3 say) and for binomial regression where all the proportions $y/n$ are well separated from 0 or 1. For other glms it either fails because $n-p$ is not the expected value of the deviance or is meaningless because the deviance needs to be scaled by the dispersion parameter. $\endgroup$
    – Gordon Smyth
    Mar 17, 2021 at 4:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.