# Logistic Regression : How to obtain a saturated model

I just read about the deviance measure for the logistic regression. However, the part that is called saturated model is not clear to me.

I did an extensive Google search but none of the results answered my question. So far I found out that a saturated model has a parameter for each observation which as a consequence results in a perfect fit. This is clear to me. But: further the fitted values (of a saturated model) are equal to the observed values.

Since from my knowledge, logistic regression is used for classification the given observed data are covariates with additional labels $y \in \{0,1\}$. However, the deviance measure employs probabilities but not the actual labels. One applies the calculated predicted probability of the logistic regression versus observed probabilities. However, since one has only given labels instead of probabilities I'm confused how to construct a saturated model from these labels?

-

## 1 Answer

For each $y_i$, the fitted probability from the saturated model will be the same as $y_i$, either zero or one. As explained here, the likelihood of the saturated model is $1$. Therefore, the deviance of such model will be $-2\log(1/1) = 0$, on $0$ df. Here is an example from R:

y = c(1,1,1,0,0,0)
a <- factor(1:length(y))
fit <- glm(y~a,family=binomial)
summary(fit)

Deviance Residuals:
0  0  0  0  0  0

Null deviance: 8.3178e+00  on 5  degrees of freedom

Residual deviance: 2.5720e-10  on 0  degrees of freedom


The saturated model always has $n$ parameters where $n$ is the sample size. That's why the null deviance is always on $(n - 1)$ df, since the null model has only the intercept. E.g., if I add one replicate for each of the six factor levels, I will get the following:

> k2
[1] 1 2 3 4 5 6 1 2 3 4 5 6
Levels: 1 2 3 4 5 6
> y2
[1] 1 1 1 0 0 0 1 1 1 0 0 0
> fit3 = glm(y2 ~ k2, family = binomial)
> summary(fit3)

Null deviance: 1.6636e+01  on 11  degrees of freedom
Residual deviance: 5.1440e-10  on  6  degrees of freedom

-
Are you sure? If $y_i$ was $0$ the denominator in the deviance expression becomes $0$ as well which is then not defined. – toom Sep 2 '14 at 19:07
Ah, yeah I see. Thanks for your answer. :) – toom Sep 2 '14 at 19:54