Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
38
questions
205
votes
10
answers
222k
views
How to deal with perfect separation in logistic regression?
If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message:
...
- 7,011
63
votes
1
answer
20k
views
Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? [duplicate]
I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
- 763
37
votes
4
answers
22k
views
Why does logistic regression become unstable when classes are well-separated?
Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
- 471
27
votes
1
answer
8k
views
Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?
We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
- 34.9k
49
votes
2
answers
151k
views
Logistic regression model does not converge
I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
- 1,199
9
votes
1
answer
2k
views
Enormous coefficients in logistic regression - what does it mean and what to do?
I get enormous coefficients during logistic regression, see coefficients with krajULKV:
...
- 6,003
10
votes
2
answers
6k
views
How to deal with quasi-complete separation in a logistic GLMM?
Update: Since I now know that my problem is called quasi-complete separation I updated the question to reflect this (thanks to Aaron).
I have a dataset from an experiment in which 29 human ...
- 13.9k
6
votes
1
answer
2k
views
High p-values for logistic regression variable that perfectly separates?
I'm using R to run some logistic regression. My variables were continuous, but I used cut to bucket the data. Some particular buckets for these variables always result in dependent variable being ...
- 771
30
votes
1
answer
30k
views
Understanding complete separation for logistic regression [duplicate]
Why does logistic regression not converge for a linearly separable data set?
For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
- 343
20
votes
1
answer
7k
views
Seeking a Theoretical Understanding of Firth Logistic Regression
I am trying to understand Firth logistic regression (method of handling perfect/complete or quasi-complete separation in logistic regression) so I can explain it to others in simplified terms. Does ...
- 415
4
votes
3
answers
600
views
Binomial GLM - non-significant difference between 100% opposite groups of observations
What follows is a basic question concerning Binomial GLM's.
Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D -
...
- 242
7
votes
1
answer
7k
views
Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?
We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings:
Warning messages:
1: glm.fit: algorithm did not ...
- 34.9k
8
votes
2
answers
2k
views
Is it possible to simulate logistic regression without randomness?
We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
- 34.9k
3
votes
2
answers
8k
views
Logistic glm with good predictors is giving p-values = 1
I have the following dataframe on which I did logistic regression with response as outcome. There are some good predictors in these variables so I expected significant variables.
...
- 455
2
votes
3
answers
5k
views
GLM high standard errors, but variables are definitely not collinear
When I use a GLM using R, my standard errors are ridiculously high. It can't be because the independent variables are related because they are all distinct ratings for an individual (i.e., interaction ...
- 21
1
vote
2
answers
1k
views
CI for logistic regression
What does it means if no CI was given for binary logistic regression analysis in SPSS output?
- 21
15
votes
2
answers
20k
views
Issue with complete separation in logistic regression (in R)
I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios.
When estimating the model in R, the ...
- 153
15
votes
3
answers
2k
views
Intuition for Support Vector Machines and the hyperplane
In my project I want to create a logistic regression model for predicting binary classification (1 or 0).
I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
- 629
11
votes
1
answer
5k
views
Binomial glmm with a categorical variable with full successes
I am running a glmm with a binomial response variable and a categorical predictor. The random effect is given by the nested design used for the data collection. The data looks like this:
...
AtiQP
8
votes
5
answers
3k
views
Why is it desirable to have linear separability in SVM?
Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ?
Can anyone please ...
- 251
6
votes
3
answers
3k
views
Is this really perfect separation in logistic regression, or is something else going on?
I have some data on patients presenting to emergency departments after sustaining self-inflicted gunshot injuries, stored in a data frame ("SIGSW," which is ~16,000 observations of 47 variables) in R. ...
- 125
5
votes
0
answers
233
views
Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect
In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
- 644
5
votes
0
answers
370
views
Separation in logistic regression in a complex survey?
Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
- 1,566
4
votes
1
answer
5k
views
If logistic regression is a linear classifier why does it fail on linearly separable data?
Logistic regression is a linear model, decision boundary generated is linear.
If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
- 143
3
votes
0
answers
718
views
Mixed logistic model with complete separation [duplicate]
I want am trying to produce a mixed logistic model but certain explanatory variables suffer from complete separation. I am aware that I need to either use exact logistic regression or a firth ...
- 1,173
3
votes
2
answers
174
views
fitting a gam with constraints on parameters to deal with separation in parametric terms
I am trying to fit a gam using mgcv which has a mix of smooth and parametric terms. The model is for some count data on fish catches. I am modelling variation in location and time, but also ...
- 41
3
votes
1
answer
754
views
How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?
I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
- 171
3
votes
2
answers
945
views
Normally distributed estimated parameters in logistic regression
I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
- 642
3
votes
1
answer
648
views
Handling quasi-perfect separation in a zero-inflated negative binomial regression in R
I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
- 33
3
votes
1
answer
439
views
Dealing with quasi-complete separation in General Additive Model?
I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
2
votes
1
answer
317
views
Logistic Regression assumption
In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
2
votes
1
answer
2k
views
A categorical variable in glm shows significance from analysis of deviance, but each level is not significant in z-test
I am fitting a generalized linear model (glm). The explanatory variable is categorical with three levels (control, treat1, treat2). The response variable is 0 or 1.
The response rate for each ...
- 2,011
2
votes
1
answer
316
views
How to use propensity scores that exhibit separation
I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
- 383
2
votes
1
answer
257
views
Will the p value become useless in such case: logistic regression with perfect separation? [duplicate]
I think I understand the perfect separation problem in logistic regression and answered my own question in this post from optimization perspective.
Is there any intuitive explanation of why logistic ...
- 34.9k
2
votes
1
answer
5k
views
Multicollinearity and perfect separation in logistic regression: what should I do?
I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
- 21
1
vote
1
answer
2k
views
I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message
I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do:
...
- 45
1
vote
0
answers
40
views
How can all of these categories be insignificant in my logistic regression [duplicate]
(All of the following is done in R, code to reproduce the dataset is given at the end of this post.)
I have a simulated data set, generated in the following way:
Make 10 categories and label them 1-...
- 4,020
0
votes
1
answer
4k
views
Logistic Regression: p values all '1', yet model fits perfectly [duplicate]
I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation.
Here's some economic data. All we have to do is create a model to predict whether the ...
- 29