Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they are in the same scale as the DV and very clearly the difference between y and the y predicted by the model. However for logistic regression, in the past I've typically just examined estimates of model fit, e.g. AIC, because I wasn't sure what a residual would mean for a logistic regression. After looking into R's help files a little bit I see that in R there are five types of glm residuals available, c("deviance", "pearson", "working","response", "partial"). The help file refers to Davison, A. C. and Snell, E. J. (1991) Residuals and diagnostics. In: Statistical Theory and Modelling. In Honour of Sir David Cox, FRS, eds. Hinkley, D. V., Reid, N. and Snell, E. J., Chapman & Hall, of which I do not have a copy. Is there a short way to describe how to interpret each of these types? In a logistic context will sum of squared residuals provide a meaningful measure of model fit or is one better off with an Information Criterion?

share|improve this question
There are elements to this question that remain unanswered, e.g. the nature of the "pearson", "working","response", and "partial" residuals, but for now I will accept Thylacoleo's answer. – rpierce Aug 12 '10 at 7:12
One really easy way to check model fit is a plot of the observed vs the predicted proportions. But this won't work if you have bernoulli regression (i.e. all of your observations have unique combinations of the independent variables, so that $n_i=1$), because you will just see a line of zeros and ones. – probabilityislogic Feb 17 '11 at 13:17
Yeah - sadly I usually am using a Bernoulli DV. – rpierce Feb 17 '11 at 20:05
Some explanation and definition of Pearson and Deviance and studentized residuals available at – Marco Stamazza Jan 11 at 9:52
@rpierce: Then you can simply overplot a smooth (such as loess) of the 0's and 1's. – kjetil b halvorsen Jan 11 at 11:17
up vote 21 down vote accepted

The easiest residuals to understand are the deviance residuals as when squared these sum to -2 times the log-likelihood. In its simplest terms logistic regression can be understood in terms of fitting the function $p = \text{logit}^{-1}(X\beta)$ for known $X$ in such a way as to minimise the total deviance, which is the sum of squared deviance residuals of all the data points.

The (squared) deviance of each data point is equal to (-2 times) the logarithm of the difference between its predicted probability $\text{logit}^{-1}(X\beta)$ and the complement of its actual value (1 for a control; a 0 for a case) in absolute terms. A perfect fit of a point (which never occurs) gives a deviance of zero as log(1) is zero. A poorly fitting point has a large residual deviance as -2 times the log of a very small value is a large number.

Doing logistic regression is akin to finding a beta value such that the sum of squared deviance residuals is minimised.

This can be illustrated with a plot, but I don't know how to upload one.

share|improve this answer
Reg images: Use one of the free image hosting sites (search google) , upload the plot to that site and link it here. – user28 Aug 10 '10 at 1:05
I've corrected an error in my original answer. I first wrote p=logit(Xbeta). In fact the predicted probability is the inverse logit of the linear combination, p=inv-logit(Xbeta). In R this is calculated as p<-plogit(Xbeta), which is p=exp(Xbeta)/(1+exp(X*beta)). – Thylacoleo Aug 10 '10 at 1:07
Which R package is plogit from? It wasn't clear if you were defining it here or getting it from somewhere else. – Amyunimus Jan 31 '13 at 18:54
@Amyunimus plogit is in R (stats), no package required (at least not anymore) – rpierce Jul 7 '14 at 7:26

I find the binnedplot function in the R package arm gives a very helpful plot of residuals. It's described nicely on p.97-101 of Gelman and Hill 2007.

share|improve this answer

On Pearsons residuals,

The Pearson residual is the difference between the observed and estimated probabilities divided by the binomial standard deviation of the estimated probability. Therefore standardizing the residuals. For large samples the standardized residuals should have a normal distribution.

From Menard, Scott (2002). Applied logistic regression analysis, 2nd Edition. Thousand Oaks, CA: Sage Publications. Series: Quantitative Applications in the Social Sciences, No. 106. First ed., 1995. See Chapter 4.4

share|improve this answer
this is not entirely correct about large samples. It is rather that you require large binomial cell counts $n_i$, or what is the same thing, a large amount of replication of covariates. The pearson residuals are far from normally distributed for any observation where $n_i<5$. – probabilityislogic Nov 24 '11 at 9:26

This book by Hardin and Hilbe, available on Google Books, provides neat short explanations of the various types of residuals.

share|improve this answer

The working residuals are the residuals in the final iteration of any iteratively weighted least squares method. I reckon that means the residuals when we think its the last iteration of our running of model. That can give rise to discussion that model running is an iterative exercise.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.