Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

Filter by
Sorted by
Tagged with
2
votes
2answers
3k views

Binary logistic regression with multiple independent variables

I have a group of 196 patients. I want to know if infection (the outcome, or dependent variable) depends on other variables. I am running a binary logistic regression with 8 independent variables (age,...
0
votes
0answers
856 views

fit warning - matched pairs logistic regression

I'm using matched pairs logistic regression (1-1 matched case-control; Hosmer and Lemeshow 2000) to model differences between vegetation selected at nest sites vs. paired random sites. To do this, I ...
4
votes
1answer
3k views

What is the exact difference between linearly separable and non-linearly separable data points?

Does the separating boundary of a given set of points have to be a straight line (or a flat hyperplane)? The image below seems to be clearly linearly separable. The other one here (the classic XOR) ...
1
vote
0answers
524 views

Knowing what exogenous variable cause quasi-separation

I'm trying to build a classifier using a logistic regression and statsmodel is telling me that there is an issue of quasi-separation. Well this isn't an issue! This is exactly what i'm trying to do: ...
1
vote
0answers
38 views

How can all of these categories be insignificant in my logistic regression [duplicate]

(All of the following is done in R, code to reproduce the dataset is given at the end of this post.) I have a simulated data set, generated in the following way: Make 10 categories and label them 1-...
1
vote
0answers
346 views

How to detect perfect separation of logistic regression? [duplicate]

I have same error message as in this post. However all my coefficients look normal with no inflated value or standard errors. My question is how can I make sure the error message is a sign of perfect ...
0
votes
1answer
1k views

SLP vs. MLP: Is my data linearly separable?

I implemented an artificial neural network using scikit neuralnetwork. As default configuration for my classification task I am using 10730 Datsets x 115 Features 1 Hidden Layer with 61 neurons 7 ...
1
vote
1answer
655 views

Complete separation and stepwise regression - possible in R?

I've been using stepAIC to narrow down my logistic regression model. However, I get the following warning when I run my model: glm.fit: fitted probabilities numerically 0 or 1 occurred I know this ...
1
vote
0answers
130 views

logistic regression - complete/quasi-separation [duplicate]

What is the implication if I don't fix a logistic regression that has complete or quasi separation? can I still read the marginal effects or are they not going to be valid? My exercise is actually ...
0
votes
2answers
616 views

logistic regression simulation doesn't converge [duplicate]

I want to do a logistic regression simulation using R I use this code ...
4
votes
1answer
307 views

Quasi-complete separation

I have a question regarding quasi-complete separated data. One example of quasi-complete separation is a dataset, where all x < 2 have y=1, all x > 2 have y=0 and some x=2 have y=0 and some x=2 ...
1
vote
1answer
6k views

How to deal with clmm warning: “hessian is numerically singular”?

I am using R's ordinal package to run a mixed regression model with an ordinal dependent variable. The data I am working with looks like this: ...
1
vote
0answers
292 views

Why logistic regression functions do not produce the right decision boundary?

I created some data using the following code: ...
1
vote
1answer
5k views

Very Confused: Getting AUC of 1 and 100% accuracy for classification task

I am building a healthcare readmission model. It is a binary classification task. I had around 90K observations with close 500 features. Except 9-10 features, rest all are binary features. I did 5 ...
0
votes
0answers
311 views

How to deal with a model that can't be fit due to quasi-complete separation? [duplicate]

I am doing a binary logistic regression analysis. I got one categorical predictor with 7 levels. When I try to do this in Minitab 17 I get an error message: "The model could not be fit. Maximum ...
3
votes
0answers
597 views

Handling singular matrix / linear separation in multinomial logit regression

I am doing an analysis of a choice-based conjoint / discrete choice experiment. After using the mlogit.data function in the ...
2
votes
2answers
6k views

Logistic glm with good predictors is giving p-values = 1

I have the following dataframe on which I did logistic regression with response as outcome. There are some good predictors in these variables so I expected significant variables. ...
6
votes
3answers
3k views

Is this really perfect separation in logistic regression, or is something else going on?

I have some data on patients presenting to emergency departments after sustaining self-inflicted gunshot injuries, stored in a data frame ("SIGSW," which is ~16,000 observations of 47 variables) in R. ...
2
votes
1answer
2k views

Using conditional logistic regression for repeated measures, complete separation (and secondarily, proc logistic)

I'm measuring a single binary outcome, with independent variables: 1) Treatment versus control. Each participant is one or the other. 2) "Before" versus "after" -- each participant has their outcome ...
2
votes
3answers
4k views

GLM high standard errors, but variables are definitely not collinear

When I use a GLM using R, my standard errors are ridiculously high. It can't be because the independent variables are related because they are all distinct ratings for an individual (i.e., interaction ...
1
vote
1answer
61 views

SVM - running time for detecting if data is linearly separable?

If my understanding is correct, one way to check if a set of $m$ data points is linearly separable is to use support vector machines to find a maximum margin hyperlane for separating the data; the ...
-3
votes
1answer
117 views

what's wrong with my data? [closed]

Sorry,owing to my reputation,I have to delete the above word. Originally I just want to copy this page's method,the author use titanic data to analyze relationship between fare and survivor. And I ...
1
vote
0answers
72 views

Logistic regression, separating variable (moderator) true in population! [duplicate]

I already checked other posts in this area, but still couldn't get a fit to my issue: I have the following preconditions: Software: preferred SPSS v21, possibly R Sample size: 5655 (will get around ...
1
vote
1answer
31 views

Unconnected Linearly Seperable Classification

Consider classifying something like the case shown below (exagerated syntetic example): If this were a task to classsify into 3 groups, (blue-left, red, blue-right), then a Linear Support Vector ...
4
votes
3answers
460 views

How to deal with the “sure probability” (p=1) in logistic regression

The logistic regression model is: $$\log\bigg(\frac{p}{1-p}\bigg) = \ldots$$ The most interesting case (for me) is the case that we have $p=1$ and $p=0$. But in this case, the ratio $p/(1-p)$ doesn'...
1
vote
0answers
1k views

Probit model: marginal effects cannot be estimated because one dummy variable was dropped for predicting failure perfectly

I have a basic question about the -margins- command in Stata: I was wondering if there was a workaround to run marginal effects for a model where one of the dummy ...
1
vote
1answer
4k views

Large value of exp (B) in binary logistic regression SPSS what is wrong? [duplicate]

I had a very large value for Exp(B) in SPSS binary logistic regression. What is wrong and what should I do?
0
votes
0answers
33 views

Logit model - none of the cases vary on one predictor

What do you do with a categorical predictor (e.g. Black-White-Hispanic) in a logistic model when none of the cases are White, but about half the population studied is? You have to drop this predictor ...
24
votes
1answer
551 views

What is the probability that $n$ random points in $d$ dimensions are linearly separable?

Given $n$ data points, each with $d$ features, $n/2$ are labeled as $0$, the other $n/2$ are labeled as $1$. Each feature takes a value from $[0,1]$ randomly (uniform distribution). What's the ...
0
votes
0answers
984 views

Trained Logistic Regression returns 'NAN' for some out of sample data

I'm using MATLAB R2015a, glmfit function for training and glmval for out of sample ...
1
vote
1answer
760 views

perfect variable separation, determine cutoff via ROCR package in R

I am developing a logistic regression model where perfect variable separation occurs. I want to calculate a cutoff from this data. Interestingly, the length of the slot ...
0
votes
0answers
1k views

Using results of Canonical Discriminant Analysis to get overall variable importance?

I have a dataset with thousands of observations pre-assigned to 18 groups and with measures for 8 different variables. I am using canonical discriminant analysis to see how separable my 18 groups are. ...
1
vote
0answers
379 views

How to deal with separation in logistic regression? [duplicate]

I'm running a binary logistic regression on 15 independent variables for 180 observations in Stata (version 11). This I do for four different groups, i.e. four dependent variables. For three, it works ...
11
votes
1answer
4k views

Binomial glmm with a categorical variable with full successes

I am running a glmm with a binomial response variable and a categorical predictor. The random effect is given by the nested design used for the data collection. The data looks like this: ...
3
votes
0answers
709 views

Mixed logistic model with complete separation [duplicate]

I want am trying to produce a mixed logistic model but certain explanatory variables suffer from complete separation. I am aware that I need to either use exact logistic regression or a firth ...
4
votes
2answers
530 views

2D projection to maximise separability

I have a set of 500 points in 5D. Each point belongs to one of five classes, and the class labels are known. I’d like to visualise the dataset in 2D such that the classes would be separated as much ...
5
votes
1answer
1k views

High p-values for logistic regression variable that perfectly separates?

I'm using R to run some logistic regression. My variables were continuous, but I used cut to bucket the data. Some particular buckets for these variables always result in dependent variable being ...
15
votes
1answer
6k views

Seeking a Theoretical Understanding of Firth Logistic Regression

I am trying to understand Firth logistic regression (method of handling perfect/complete or quasi-complete separation in logistic regression) so I can explain it to others in simplified terms. Does ...
21
votes
1answer
2k views

Model selection with Firth logistic regression

In a small data set ($n\sim100$ ) that I am working with, several variables give me perfect prediction/separation. I thus use Firth logistic regression to deal with the issue. If I select the best ...
8
votes
1answer
3k views

How to describe and present the issue of perfect separation?

Folks who work with logistic regression are familiar with the issue of perfect separation: if you have a variable specific values of which are associated with only one of the two outcomes (say a ...
6
votes
1answer
2k views

Perfect separation error message for glm with binomial but not with quasibinomial family

I am attempting to create a model which looks at the effect that age, supplementary food use, and nest initiation date (converted to Julian days) is having on female reproductive success (success =1 ...
1
vote
1answer
1k views

A categorical variable in glm shows significance from analysis of deviance, but each level is not significant in z-test

I am fitting a generalized linear model (glm). The explanatory variable is categorical with three levels (control, treat1, treat2). The response variable is 0 or 1. The response rate for each ...
3
votes
2answers
1k views

What causes perfect prediction but no significant predictors in logistic regression?

I want to do a logistic regression with R. I have 18 continuous covariates and a sample consisting of 100 observations. When I enter all of the covariates into the ...
5
votes
0answers
267 views

Separation in logistic regression in a complex survey?

Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
9
votes
1answer
1k views

Enormous coefficients in logistic regression - what does it mean and what to do?

I get enormous coefficients during logistic regression, see coefficients with krajULKV: ...
1
vote
2answers
939 views

CI for logistic regression

What does it means if no CI was given for binary logistic regression analysis in SPSS output?
59
votes
1answer
16k views

Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what?

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
8
votes
2answers
5k views

How to deal with quasi-complete separation in a logistic GLMM?

Update: Since I now know that my problem is called quasi-complete separation I updated the question to reflect this (thanks to Aaron). I have a dataset from an experiment in which 29 human ...
187
votes
8answers
188k views

How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
8
votes
1answer
7k views

Unexpected singularities in the Hessian matrix error in multinomial logistic regression

I have been doing multinomial logistic regression analysis using SPSS 19. I have encountered the following problem when I run the analysis procedure: "Unexpected singularities in the Hessian ...