1
$\begingroup$

I have a dataset with a binomial response variable (1/0) and a single explanatory categorical variable with 2 levels. For one level, the response is all 1s and no 0s (e.g. Y1/M1) If there are some 0s in responses for both levels, the effect is estimated correctly (e.g. Y2/M2)

Why doesn't it work here when one level has all 1s? Why can't the model give a sensible estimate and SE? What would be the solution to obtain an effect estimate here? Thanks for your help

Y1 <- c(sample(c(1),50, replace = TRUE),
     sample(c(0,1),50, replace = TRUE))
Y2 <- sample(c(0,1),100, replace = TRUE)
X <- rep(c("A","B"), each = 50)

df <- data.frame(Y1,Y2,X)

M1 <- glm(Y1~X, data = df, family = binomial)
M2 <- glm(Y2~X, data = df, family = binomial)

> summary(M1)

Call:
glm(formula = Y1 ~ X, family = binomial, data = df)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-1.21159   0.00008   0.00008   1.14361   1.14361  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)    19.57    1520.85   0.013     0.99
XB            -19.49    1520.85  -0.013     0.99

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 110.216  on 99  degrees of freedom
Residual deviance:  69.235  on 98  degrees of freedom
AIC: 73.235

Number of Fisher Scoring iterations: 18

> summary(M2)

Call:
glm(formula = Y2 ~ X, family = binomial, data = df)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.0769  -1.0769  -0.8782   1.2814   1.5096  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)  -0.2412     0.2849  -0.846    0.397
XB           -0.5126     0.4160  -1.232    0.218

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 132.81  on 99  degrees of freedom
Residual deviance: 131.28  on 98  degrees of freedom
AIC: 135.28

Number of Fisher Scoring iterations: 4
$\endgroup$
0

1 Answer 1

8
$\begingroup$

What you are seeing is called quasi-complete separation in the context of binomial and logistic regression. In essence, at least one of the levels of one of your independent categorical variables is perfectly correlated with your response variable.

Complete separation is when each level of at least one or more categorical independent variables is perfectly correlated with each level of the binary response.

Since there is no variation in the response for the level that is quasi-separated, one level of a categorical independent variable can explain one of the levels of the binary response deterministically.

Numerically, quasi-complete separation results in a parameter diverging to infinity during log-likelihood optimization. Hence that parameter cannot be estimated. This means that the maximum likelihood estimates(MLEs) do not exist.

NOTE: Collinearity is a separate and distinct phenomenon, only exhibited by independent variables in your model. A model has collinear variables when the independent variables are highly correlated.

For more information on how to deal with separation, please see the following information on the Firth correction:

https://cemsiis.meduniwien.ac.at/kb/wf/software/statistische-software/firth-correction/

$\endgroup$
1
  • 2
    $\begingroup$ Well put. Also, the ordinary maximum likelihood estimate of $\infty$ is completely valid, leading to a predicted probability of 1.0. The standard error is invalid, so one has to use profile likelihood to get a valid confidence interval. $\endgroup$ Dec 11, 2021 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.