Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

Filter by
Sorted by
Tagged with
187
votes
8answers
187k views

How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
59
votes
1answer
16k views

Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what?

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
36
votes
4answers
14k views

Why does logistic regression become unstable when classes are well-separated?

Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
48
votes
2answers
127k views

Logistic regression model does not converge

I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
23
votes
1answer
6k views

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
9
votes
1answer
1k views

Enormous coefficients in logistic regression - what does it mean and what to do?

I get enormous coefficients during logistic regression, see coefficients with krajULKV: ...
21
votes
1answer
17k views

Understanding complete separation for logistic regression [duplicate]

Why does logistic regression not converge for a linearly separable data set? For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
5
votes
1answer
1k views

High p-values for logistic regression variable that perfectly separates?

I'm using R to run some logistic regression. My variables were continuous, but I used cut to bucket the data. Some particular buckets for these variables always result in dependent variable being ...
15
votes
1answer
6k views

Seeking a Theoretical Understanding of Firth Logistic Regression

I am trying to understand Firth logistic regression (method of handling perfect/complete or quasi-complete separation in logistic regression) so I can explain it to others in simplified terms. Does ...
8
votes
2answers
5k views

How to deal with quasi-complete separation in a logistic GLMM?

Update: Since I now know that my problem is called quasi-complete separation I updated the question to reflect this (thanks to Aaron). I have a dataset from an experiment in which 29 human ...
7
votes
1answer
5k views

Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
8
votes
2answers
2k views

Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
3
votes
2answers
215 views

Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
2
votes
2answers
6k views

Logistic glm with good predictors is giving p-values = 1

I have the following dataframe on which I did logistic regression with response as outcome. There are some good predictors in these variables so I expected significant variables. ...
1
vote
2answers
938 views

CI for logistic regression

What does it means if no CI was given for binary logistic regression analysis in SPSS output?
12
votes
1answer
10k views

Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
15
votes
3answers
2k views

Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
11
votes
1answer
4k views

Binomial glmm with a categorical variable with full successes

I am running a glmm with a binomial response variable and a categorical predictor. The random effect is given by the nested design used for the data collection. The data looks like this: ...
2
votes
3answers
4k views

GLM high standard errors, but variables are definitely not collinear

When I use a GLM using R, my standard errors are ridiculously high. It can't be because the independent variables are related because they are all distinct ratings for an individual (i.e., interaction ...
6
votes
3answers
3k views

Is this really perfect separation in logistic regression, or is something else going on?

I have some data on patients presenting to emergency departments after sustaining self-inflicted gunshot injuries, stored in a data frame ("SIGSW," which is ~16,000 observations of 47 variables) in R. ...
3
votes
0answers
709 views

Mixed logistic model with complete separation [duplicate]

I want am trying to produce a mixed logistic model but certain explanatory variables suffer from complete separation. I am aware that I need to either use exact logistic regression or a firth ...
3
votes
1answer
105 views

Dealing with quasi-complete separation in General Additive Model?

I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
1
vote
1answer
92 views

Logistic Regression assumption

In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
0
votes
1answer
2k views

Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
8
votes
4answers
2k views

Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
5
votes
0answers
266 views

Separation in logistic regression in a complex survey?

Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
3
votes
1answer
554 views

How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
3
votes
2answers
421 views

Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
2
votes
1answer
245 views

Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
2
votes
1answer
153 views

Will the p value become useless in such case: logistic regression with perfect separation?

I think I understand the perfect separation problem in logistic regression and answered my own question in this post from optimization perspective. Is there any intuitive explanation of why logistic ...
1
vote
1answer
1k views

A categorical variable in glm shows significance from analysis of deviance, but each level is not significant in z-test

I am fitting a generalized linear model (glm). The explanatory variable is categorical with three levels (control, treat1, treat2). The response variable is 0 or 1. The response rate for each ...
1
vote
0answers
38 views

How can all of these categories be insignificant in my logistic regression [duplicate]

(All of the following is done in R, code to reproduce the dataset is given at the end of this post.) I have a simulated data set, generated in the following way: Make 10 categories and label them 1-...