Logistic regression and error terms

Question

In logistic regression, if we considered residuals, could they only take on the values $0$ or $1$? The data points themselves take on only $1$ or $0$. The logistic curve can take on any value between $0$ and $1$. What would the distribution of the residuals look like?

Answer 1

There are many kinds of residuals. This is true of linear regression (e.g., raw, studentized, standardized, etc.), but is even more true of logistic regression. R has options available for retrieving 5 different types of residuals from a logistic regression fit (see ?residuals.glm: type = c("deviance", "pearson", "working", "response", "partial").

You are right that the response variable in a logistic regression can take on only $0$ and $1$, and that logistic regression's predicted values $\hat Y$ can take on any real value within $(0, 1)$. However, that implies that raw residuals, $\hat y_i - y_i$, will take on real values within the interval $(0, 1)$ as well. Moreover, they can take on many different values unless the $X$ values are limited to a few discrete levels and/or the response is completely unrelated to $X$, not only in the population, but in your sample.

These facts pertain to raw residuals, though, which are basically not used by anyone.
The most common residuals to examine in logistic regression are deviance residuals. Their calculation is very hard to understand (it is listed in the link @whuber gave), but they can be roughly normally distributed with sufficient variability in $X$. Below, I do a simple demonstration in R to show the deviance residuals from an experimental situation with two groups and an observational situation where $X$ is uniformly distributed.

lo.2.p = function(lo){              # this function will convert log odds to 
  o = exp(lo)                       #   probabilities
  p = o/(o+1)
  return(p)
}

set.seed(9044)                      # this makes the example exactly reproducible
x.e = rep(c(0, 20), each=20)        # in the experimental case, there are 2 groups
x.o = runif(40, min=-20, max=20)    # for observational, X is uniform
p.e = lo.2.p(-.2 + .3*x.e)          # on the log odds scale, the intercept is -.2,
p.o = lo.2.p(-.2 + .3*x.o)          #   & the slope is .3
y.e = rbinom(40, size=1, prob=p.e)  # this generates the response data
y.o = rbinom(40, size=1, prob=p.o)

mod.e = glm(y.e~x.e)                # here I fit the 2 models
mod.o = glm(y.o~x.o)

Answer 2

There are no error terms in logistic regression. And in my opinion the only residuals that are useful are partial residuals useful for checking departures from the assumed shape of regression effects. But even that can be dealt with more directly using regression splines in the first place. There are no intrinsic residuals for the model.