Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also?
Null deviance: 1146.1 on 1077 degrees of freedom Residual deviance: 4589.4 on 1099 degrees of freedom AIC: 11089
Let LL = loglikelihood
Here is a quick summary of what you see from the summary(glm.fit) output,
Null Deviance = 2(LL(Saturated Model) - LL(Null Model)) on df = df_Sat - df_Null
Residual Deviance = 2(LL(Saturated Model) - LL(Proposed Model)) df = df_Sat - df_Proposed
The Saturated Model is a model that assumes each data point has its own parameters (which means you have n parameters to estimate.)
The Null Model assumes the exact "opposite", in that is assumes one parameter for all of the data points, which means you only estimate 1 parameter.
The Proposed Model assumes you can explain your data points with p parameters + an intercept term, so you have p+1 parameters.
If your Null Deviance is really small, it means that the Null Model explains the data pretty well. Likewise with your Residual Deviance.
What does really small mean? If your model is "good" then your Deviance is approx Chi^2 with (df_sat - df_model) degrees of freedom.
If you want to compare you Null model with your Proposed model, then you can look at
(Null Deviance - Residual Deviance) approx Chi^2 with df Proposed - df Null = (n-(p+1))-(n-1)=p
Are the results you gave directly from R? They seem a little bit odd, because generally you should see that the degrees of freedom reported on the Null are always higher than the degrees of freedom reported on the Residual. That is because again:
Null_Deviance_df = Saturated_df - Null_df = n-1
Residual_Deviance_df = Saturated_df - Proposed_df = n-(p+1)
GLM?
$\endgroup$
The null deviance shows how well the response is predicted by the model with nothing but an intercept.
The residual deviance shows how well the response is predicted by the model when the predictors are included. From your example, it can be seen that the deviance goes up by 3443.3 when 22 predictor variables are added (note: degrees of freedom = no. of observations – no. of predictors) . This increase in deviance is evidence of a significant lack of fit.
We can also use the residual deviance to test whether the null hypothesis is true (i.e. Logistic regression model provides an adequate fit for the data). This is possible because the deviance is given by the chi-squared value at a certain degrees of freedom. In order to test for significance, we can find out associated p-values using the below formula in R:
p-value = 1 - pchisq(deviance, degrees of freedom)
Using the above values of residual deviance and DF, you get a p-value of approximately zero showing that there is a significant lack of evidence to support the null hypothesis.
> 1 - pchisq(4589.4, 1099)
[1] 0
While both answers given here are correct (and really useful resources), from page 432 of Introduction to Linear Regression Analysis (Montgomery, Peck, Vining, 5E), a general rule of thumb is given as if $$ \frac{D}{n-p} >> 1, $$ where $p$ is the number of regressors, $n$ is the number of observations and $D$ is the residual deviance, then the fit can be considered inadequate.
