# Residuals for Logistic Regression

I have read that there are multiple ways to look at residuals for logistic regression. I am not necessarily that interested in analyzing the residuals for a logit model, but my main question is rather:

Where are residuals in the specification of the model?

For linear models, it's common to write $$y=X\beta+\varepsilon \ \ \text{ or } \ \ y = \beta_0+\beta_1x_1 + \dots + \beta_kx_k+\varepsilon$$

However, for the logit model I keep finding $$l=\log\frac{p}{1-p}=\beta_0+\beta_1x_1 + \dots + \beta_kx_k \ \ \text{ or } \ \ p = \frac{1}{1+e^{-(\beta_0+\beta_1x_1 + \dots + \beta_kx_k)}}$$

How come the disturbances are not part of the model in this case?

Edit:

I have thought of something but am not sure if this is the complete reasoning behind it. In my understanding, a linear model assumes that data have been generated according to $$y=X\beta$$ with random disturbances, as nothing is perfectly linear.

And for a logit model this disturbance is not needed, as the data are assumed to be generated(?) according to $$p$$: at every point $$x_i$$ there is a certain probability $$p$$ that the outcome will be $$1$$, and the points $$y=0,1$$ are generated according to this, rather than a set linear relation with disturbances that make it random.

• For logistic regression, the role of $R^2$ is taken by "Pseudo $R^2$", for which different definitions are in use. All have in common that they are not based on residuals but on ratios between likelihood values. If you search for "pseudo r squared" and "logistic regression", you should find some possible definitions. Jun 27 '21 at 19:43
• You should distinguish error i.e. $\epsilon_i = Y_i-\mu_i(X_i, \beta)$ (where $\mu_i$ is a conditional mean that depends on $X_i$ and parameter $\beta$ and is unique), from residuals i.e. $Y_i-\mu_i(X_i, \hat{\beta})$ (with estimated $\beta$, which will be different each data set). Both always exist, but $var(\epsilon) = \sigma^2$ won't be parameterized when it's already determined by the mean e.g. in logistic regression's Bernoulli distrib. assumption. That linear regression requires Normality and therefore $\mu_i$ and $\sigma^2$ makes it necessary to write $\epsilon$ into the model. Jun 27 '21 at 20:47

You can have residuals, or not, in either case. But to start with, you don't need residuals at all to specify the model: In either case, you can write $$Y|X=x \ \ \sim \ \ p(y|x),$$ where $$p(y|x)$$ is some conditional distribution. In the case of logistic regression, $$p(y|x)$$ is a Bernoulli distribution. In the case of classical regression, $$p(y|x)$$ is a normal distribution, but this can be relaxed to allow other forms, like Poisson, negative binomial, multinomial, non- or semi-parametric variants, etc.
If you want residuals, you can also have them, in either case: For an observation where $$X=x$$, the residual is $$\epsilon = Y - E(Y|X=x)$$. In the case of standard regression, under the usual linearity assumption, this gives you $$\epsilon = Y - (\beta_0 + \beta_1 x).$$ In the case of logistic regression, under the usual linearity (of logit) assumption, this gives you $$\epsilon = Y - \frac{\exp{(\beta_0 + \beta_1 x)}}{1+\exp{(\beta_0 + \beta_1 x)}},$$ where $$Y$$ is either 0 or 1.
It is just not common to use this form in the case of logistic regression because the salient feature of the model is the form of the conditional Bernoulli distribution of $$Y|X=x$$.
• Although this answer is correct, it is formulated in such abstract terms that the original poster presumably won't understand it. I would suggest to add an explanation how $E(Y|X=x)$ is computed from the estimated coefficients $\beta_i$. Jun 27 '21 at 19:40