As of May 31, 2023, we have updated our Code of Conduct.

# Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

38 questions
Filter by
Sorted by
Tagged with
222k views

### How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
• 7,011
20k views

### Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? [duplicate]

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
• 763
22k views

### Why does logistic regression become unstable when classes are well-separated?

Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
• 471
8k views

### Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
• 34.9k
151k views

### Logistic regression model does not converge

I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
• 1,199
2k views

### Enormous coefficients in logistic regression - what does it mean and what to do?

I get enormous coefficients during logistic regression, see coefficients with krajULKV: ...
• 6,003
6k views

### How to deal with quasi-complete separation in a logistic GLMM?

Update: Since I now know that my problem is called quasi-complete separation I updated the question to reflect this (thanks to Aaron). I have a dataset from an experiment in which 29 human ...
• 13.9k
2k views

### High p-values for logistic regression variable that perfectly separates?

I'm using R to run some logistic regression. My variables were continuous, but I used cut to bucket the data. Some particular buckets for these variables always result in dependent variable being ...
• 771
30k views

### Understanding complete separation for logistic regression [duplicate]

Why does logistic regression not converge for a linearly separable data set? For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
• 343
7k views

### Seeking a Theoretical Understanding of Firth Logistic Regression

I am trying to understand Firth logistic regression (method of handling perfect/complete or quasi-complete separation in logistic regression) so I can explain it to others in simplified terms. Does ...
• 415
600 views

### Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
7k views

### Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
• 34.9k
2k views

### Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
• 34.9k
8k views

### Logistic glm with good predictors is giving p-values = 1

I have the following dataframe on which I did logistic regression with response as outcome. There are some good predictors in these variables so I expected significant variables. ...
• 455
5k views

### GLM high standard errors, but variables are definitely not collinear

When I use a GLM using R, my standard errors are ridiculously high. It can't be because the independent variables are related because they are all distinct ratings for an individual (i.e., interaction ...
1 vote
1k views

### CI for logistic regression

What does it means if no CI was given for binary logistic regression analysis in SPSS output?
20k views

### Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
• 153
2k views

### Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
• 629
5k views

### Binomial glmm with a categorical variable with full successes

I am running a glmm with a binomial response variable and a categorical predictor. The random effect is given by the nested design used for the data collection. The data looks like this: ...
3k views

### Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
• 251
3k views

### Is this really perfect separation in logistic regression, or is something else going on?

I have some data on patients presenting to emergency departments after sustaining self-inflicted gunshot injuries, stored in a data frame ("SIGSW," which is ~16,000 observations of 47 variables) in R. ...
• 125
233 views

### Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect

In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
370 views

### Separation in logistic regression in a complex survey?

Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
• 1,566
5k views

### If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
• 143
718 views

### Mixed logistic model with complete separation [duplicate]

I want am trying to produce a mixed logistic model but certain explanatory variables suffer from complete separation. I am aware that I need to either use exact logistic regression or a firth ...
• 1,173
174 views

### fitting a gam with constraints on parameters to deal with separation in parametric terms

I am trying to fit a gam using mgcv which has a mix of smooth and parametric terms. The model is for some count data on fish catches. I am modelling variation in location and time, but also ...
• 41
754 views

### How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
• 171
945 views

### Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
• 642
648 views

### Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
439 views

### Dealing with quasi-complete separation in General Additive Model?

I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
317 views

### Logistic Regression assumption

In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
2k views

### A categorical variable in glm shows significance from analysis of deviance, but each level is not significant in z-test

I am fitting a generalized linear model (glm). The explanatory variable is categorical with three levels (control, treat1, treat2). The response variable is 0 or 1. The response rate for each ...
• 2,011
316 views

### How to use propensity scores that exhibit separation

I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
• 383
257 views

### Will the p value become useless in such case: logistic regression with perfect separation? [duplicate]

I think I understand the perfect separation problem in logistic regression and answered my own question in this post from optimization perspective. Is there any intuitive explanation of why logistic ...
• 34.9k
5k views

### Multicollinearity and perfect separation in logistic regression: what should I do?

I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
• 21
1 vote
2k views

### I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
1 vote
40 views

### How can all of these categories be insignificant in my logistic regression [duplicate]

(All of the following is done in R, code to reproduce the dataset is given at the end of this post.) I have a simulated data set, generated in the following way: Make 10 categories and label them 1-...
• 4,020