Interpretation of plot (glm.model)

Question

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.

Answer 1

R does not have a distinct plot.glm() method. When you fit a model with glm() and run plot(), it calls ?plot.lm, which is appropriate for linear models (i.e., with a normally distributed error term).

In general, the meaning of these plots (at least for linear models) can be learned in various existing threads on CV (e.g.: Residuals vs. Fitted; qq-plots in several places: 1, 2, 3; Scale-Location; Residuals vs Leverage). However, those interpretations are not generally valid when the model in question is a logistic regression.

More specifically, the plots will often 'look funny' and lead people to believe that there is something wrong with the model when it is perfectly fine. We can see this by looking at those plots with a couple of simple simulations where we know the model is correct:

  # we'll need this function to generate the Y data:
lo2p = function(lo){ exp(lo)/(1+exp(lo)) }

set.seed(10)                    # this makes the simulation exactly reproducible
x  = runif(20, min=0, max=10)   # the X data are uniformly distributed from 0 to 10
lo = -3 + .7*x                  # this is the true data generating process
p  = lo2p(lo)                   # here I convert the log odds to probabilities
y  = rbinom(20, size=1, prob=p) # this generates the Y data

mod = glm(y~x, family=binomial) # here I fit the model
summary(mod)                    # the model captures the DGP very well & has no
# ...                           #  obvious problems:
# Deviance Residuals: 
#      Min        1Q    Median        3Q       Max  
# -1.76225  -0.85236  -0.05011   0.83786   1.59393  
# 
# Coefficients:
#             Estimate Std. Error z value Pr(>|z|)  
# (Intercept)  -2.7370     1.4062  -1.946   0.0516 .
# x             0.6799     0.3261   2.085   0.0371 *
# ...
# 
# Null deviance: 27.726  on 19  degrees of freedom
# Residual deviance: 21.236  on 18  degrees of freedom
# AIC: 25.236
# 
# Number of Fisher Scoring iterations: 4

Now lets look at the plots we get from plot.lm():

Both the Residuals vs Fitted and the Scale-Location plots look like there are problems with the model, but we know there aren't any. These plots, intended for linear models, are simply often misleading when used with a logistic regression model.

Let's look at another example:

set.seed(10)
x2 = rep(c(1:4), each=40)                    # X is a factor with 4 levels
lo = -3 + .7*x2
p  = lo2p(lo)
y  = rbinom(160, size=1, prob=p)

mod = glm(y~as.factor(x2), family=binomial)
summary(mod)                                 # again, everything looks good:
# ...
# Deviance Residuals: 
#   Min       1Q   Median       3Q      Max  
# -1.0108  -0.8446  -0.3949  -0.2250   2.7162  
# 
# Coefficients:
#                Estimate Std. Error z value Pr(>|z|)    
# (Intercept)      -3.664      1.013  -3.618 0.000297 ***
# as.factor(x2)2    1.151      1.177   0.978 0.328125    
# as.factor(x2)3    2.816      1.070   2.632 0.008481 ** 
# as.factor(x2)4    3.258      1.063   3.065 0.002175 ** 
# ... 
# 
# Null deviance: 160.13  on 159  degrees of freedom
# Residual deviance: 133.37  on 156  degrees of freedom
# AIC: 141.37
# 
# Number of Fisher Scoring iterations: 6

Now all the plots look strange.

So what do these plots show you?

• The Residuals vs Fitted plot can help you see, for example, if there are curvilinear trends that you missed. But the fit of a logistic regression is curvilinear by nature, so you can have odd looking trends in the residuals with nothing amiss.
• The Normal Q-Q plot helps you detect if your residuals are normally distributed. But the deviance residuals don't have to be normally distributed for the model to be valid, so the normality / non-normality of the residuals doesn't necessarily tell you anything.
• The Scale-Location plot can help you identify heteroscedasticity. But logistic regression models are pretty much heteroscedastic by nature.
• The Residuals vs Leverage can help you identify possible outliers. But outliers in logistic regression don't necessarily manifest in the same way as in linear regression, so this plot may or may not be helpful in identifying them.

The simple take home lesson here is that these plots can be very hard to use to help you understand what is going on with your logistic regression model. It is probably best for people not to look at these plots at all when running logistic regression, unless they have considerable expertise.

Answer 2

1. Residuals vs fitted - there should be no strong patterns (mild patterns are not a problem, see @gung's answer) and no outliers, residuals should be randomly distributed around zero.
2. Normal Q-Q - residuals should go around the diagonal line, i.e. should be normally distributed (see wiki for Q-Q plot). This plot helps checking if they are approximately normal.
3. Scale-location - as you can see, on Y axis there are also residuals (like in Residuals vs fitted plot), but they are scaled, so it's similar to (1), but in some cases it works better.
4. Residuals vs Leverage -- it helps to diagnose outlying cases. As in previous plots, outlying cases are numbered, but on this plot if there are any cases that are very different from the rest of the data they are plotted below thin red lines (check wiki on Cook's distance).

Read more on assumptions of regression as in many aspects there are similar (e.g. here, or tutorial on regression in R here).