Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
180
questions
2
votes
1
answer
250
views
Complete separation in logistic regression with only one direction
In this paper Dealing with Separation in Logistic Regression Models some various types of complete separation are discussed:
direction of the separation is positive if and only if $s_i = 1 \...
- 969
37
votes
4
answers
22k
views
Why does logistic regression become unstable when classes are well-separated?
Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
- 471
0
votes
1
answer
2k
views
perfect separation logistic regression [duplicate]
in continuity to the post stepwise logistic regression non significative variables(high p-values) and as demanded by matthew
this is a post explaining the data i have and the problem
in fact i have a ...
- 21
1
vote
1
answer
421
views
Are neural networks able to separate these classes?
I know that MLP can separate classes even if the decision boundary is nonlinear. However, in all cases that I've seen the boundary was just a single 'unclosed' line.
Is the MLP able to solve the ...
- 3,739
8
votes
5
answers
8k
views
When does logistic regression not work properly?
I need to find a situation in which logistic regression does not work well. Furthermore, I would like to know when a random forest might perform better than a logistic regression model.
- 131
2
votes
0
answers
925
views
What's the reason for large beta coefficient standard error and estimate in binary logistic regression? [closed]
I'm using SPSS for binary logistic regression. The result showed that one of the variables has a large beta coefficient and standard error.
Here are some results:
Can some explain to me what is the ...
- 61
1
vote
1
answer
107
views
How to pick two features that can linearly separate two classes as much as possible
I have a few dataset, each has about 120 features/dimensions and two classes (e.g. A and B) in it. Now I'd like to visualize the dataset from just two dimensions without doing any dimensionality ...
- 1,097
0
votes
0
answers
117
views
1
vote
0
answers
60
views
Logistic plot seems wrong [duplicate]
I'm a newbie to R and am hoping someone can show me the error of my ways. I'm getting what appears to be a logistic regression and plot but I wasn't expecting the slope around 25.2 to be so steep and ...
- 11
2
votes
1
answer
257
views
Will the p value become useless in such case: logistic regression with perfect separation? [duplicate]
I think I understand the perfect separation problem in logistic regression and answered my own question in this post from optimization perspective.
Is there any intuitive explanation of why logistic ...
- 34.9k
27
votes
1
answer
8k
views
Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?
We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
- 34.9k
13
votes
2
answers
4k
views
Is R's glm function useless in a big data / machine learning setting? [duplicate]
I am surprised that R’s glm will “break” (not converge with default setting) for the following “toy” example (binary classification with ~50k data, ~10 features), ...
- 34.9k
8
votes
1
answer
1k
views
How does Python Scikit Learn handle linear separation problem in logistic regression?
There are already posts about warnings from R dealing with logistic regression and linear separation such as this one. I just wanna make sure if in Python Scikit Learn this problem is all solved by ...
- 505
1
vote
1
answer
52
views
Which test should I use? Hypothesis: Porcupines prefer certain plant genotypes over others in a common garden
Hypothesis: Porcupines prefer certain plant genotypes over others in a common garden.
We visually inspected just over 2,500 trees in a common garden for evidence of herbivory on woody tissues. Each ...
- 95
2
votes
2
answers
1k
views
Variable selection for logistic regression with separated data
I have a fairly large dataset ($\approx 3 \bar{M}$ observations for a dozen candidate predictors) and I would like to perform a logistic regression on that dataset.
I have a problem of separation in ...
- 453
3
votes
1
answer
883
views
Quasi-separation when running Cox regression comparing 3 risk groups?
This is my first post here, so please bear with me! I'm comparing several biomarkers with Kaplan-Meier curves and calculating hazard ratios for different risk groups (defined by a certain, well ...
- 31
1
vote
1
answer
441
views
How can I prove if 2 datasets are separable or not?
I want to create a classification model, derived from real spectral measurements. My data is in the form of samples x features. The number of features is 424 and the number of samples is in the order ...
- 23
1
vote
0
answers
141
views
regularized generalized linear model
Since I run into complete separation with logistic regression I try to run a penalized logistic regression for a binomial response variable.
It doesn't seem to work for my data. In an example that ...
- 343
1
vote
1
answer
126
views
Can univariate analysis find the p value, HZ ratio from the 100% alive group analysis
I try to compare two groups in univariate analysis but there are some problems because one group was alive only (no dead)
30
votes
1
answer
30k
views
Understanding complete separation for logistic regression [duplicate]
Why does logistic regression not converge for a linearly separable data set?
For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
- 343
1
vote
0
answers
171
views
How to assess logistic regression & reduce computational effort with imbalanced data
I have a highly imbalanced data set (ratio 1:150) with four predictors, where two are correlated. The data can be found here, you can also see the two figures below.
I would like to use logistic ...
- 343
2
votes
1
answer
191
views
Really weird results for a logistic model - is it due to high frequency of one value on response variable?
I am trying to test whether experimental group (a vs b) influences the probability of some binary outcome, but the model results are strange.
The code I'm using:
...
- 450
1
vote
0
answers
39
views
Presenting finite sample examination of asymptotics
From statistical theory, we often obtain results such as
$\sqrt n (\theta - \hat \theta) \rightarrow_d N(0, \sigma)$
ie we have a normal limiting distribution.
Because this formula says nothing ...
- 19.3k
6
votes
3
answers
1k
views
Strange outcomes in binary logistic regression in SPSS
I did a binary logistic regression with SPSS 23 and I found some strange outcomes. This is for NOACprev until No_Prev_treatment, the last 6 variables. First of all they have very high outcomes for B, ...
- 63
2
votes
2
answers
5k
views
Binary logistic regression with multiple independent variables
I have a group of 196 patients. I want to know if infection (the outcome, or dependent variable) depends on other variables. I am running a binary logistic regression with 8 independent variables (age,...
- 21
0
votes
0
answers
1k
views
fit warning - matched pairs logistic regression
I'm using matched pairs logistic regression (1-1 matched case-control; Hosmer and Lemeshow 2000) to model differences between vegetation selected at nest sites vs. paired random sites. To do this, I ...
- 235
4
votes
1
answer
4k
views
What is the exact difference between linearly separable and non-linearly separable data points?
Does the separating boundary of a given set of points have to be a straight line (or a flat hyperplane)?
The image below seems to be clearly linearly separable.
The other one here (the classic XOR) ...
- 1,032
2
votes
0
answers
1k
views
Knowing what exogenous variable cause quasi-separation
I'm trying to build a classifier using a logistic regression and statsmodel is telling me that there is an issue of quasi-separation.
Well this isn't an issue! This is exactly what i'm trying to do: ...
- 31
1
vote
0
answers
40
views
How can all of these categories be insignificant in my logistic regression [duplicate]
(All of the following is done in R, code to reproduce the dataset is given at the end of this post.)
I have a simulated data set, generated in the following way:
Make 10 categories and label them 1-...
- 4,020
1
vote
0
answers
533
views
How to detect perfect separation of logistic regression? [duplicate]
I have same error message as in this post. However all my coefficients look normal with no inflated value or standard errors. My question is how can I make sure the error message is a sign of perfect ...
- 307
0
votes
1
answer
2k
views
SLP vs. MLP: Is my data linearly separable?
I implemented an artificial neural network using scikit neuralnetwork. As default configuration for my classification task I am using
10730 Datsets x 115 Features
1 Hidden Layer with 61 neurons
7 ...
- 103
1
vote
1
answer
846
views
Complete separation and stepwise regression - possible in R?
I've been using stepAIC to narrow down my logistic regression model. However, I get the following warning when I run my model:
glm.fit: fitted probabilities numerically 0 or 1 occurred
I know this ...
- 11
1
vote
0
answers
149
views
logistic regression - complete/quasi-separation [duplicate]
What is the implication if I don't fix a logistic regression that has complete or quasi separation? can I still read the marginal effects or are they not going to be valid?
My exercise is actually ...
- 19
0
votes
2
answers
1k
views
logistic regression simulation doesn't converge [duplicate]
I want to do a logistic regression simulation using R
I use this code
...
- 31
4
votes
1
answer
617
views
Quasi-complete separation
I have a question regarding quasi-complete separated data.
One example of quasi-complete separation is a dataset, where all x < 2 have y=1, all x > 2 have y=0 and some x=2 have y=0 and some x=2 ...
- 41
1
vote
1
answer
8k
views
How to deal with clmm warning: "hessian is numerically singular"?
I am using R's ordinal package to run a mixed regression model with an ordinal dependent variable. The data I am working with looks like this:
...
- 23
1
vote
0
answers
358
views
Why logistic regression functions do not produce the right decision boundary?
I created some data using the following code:
...
- 627
2
votes
1
answer
8k
views
Very Confused: Getting AUC of 1 and 100% accuracy for classification task
I am building a healthcare readmission model. It is a binary classification task. I had around 90K observations with close 500 features. Except 9-10 features, rest all are binary features.
I did 5 ...
- 1,025
0
votes
0
answers
367
views
How to deal with a model that can't be fit due to quasi-complete separation? [duplicate]
I am doing a binary logistic regression analysis. I got one categorical predictor with 7 levels. When I try to do this in Minitab 17 I get an error message: "The model could not be fit. Maximum ...
- 53
3
votes
0
answers
926
views
Handling singular matrix / linear separation in multinomial logit regression
I am doing an analysis of a choice-based conjoint / discrete choice experiment. After using the mlogit.data function in the ...
- 255
3
votes
2
answers
8k
views
Logistic glm with good predictors is giving p-values = 1
I have the following dataframe on which I did logistic regression with response as outcome. There are some good predictors in these variables so I expected significant variables.
...
- 455
30
votes
3
answers
38k
views
How to know whether the data is linearly separable?
The data has many features (e.g. 100) and the number of instances is like 100,000. The data is sparse. I want to fit the data using logistic regression or svm. How do I know whether features are ...
- 523
6
votes
3
answers
3k
views
Is this really perfect separation in logistic regression, or is something else going on?
I have some data on patients presenting to emergency departments after sustaining self-inflicted gunshot injuries, stored in a data frame ("SIGSW," which is ~16,000 observations of 47 variables) in R. ...
- 125
2
votes
1
answer
607
views
Interpreting very small Exp(B) in multinomial logistic regression
I am having trouble interpreting the Exp(B) value in a multinomial logistic regression in which my outcome variable is categorical (3 categories) and my predictor is a scale variable. The Exp(B) value ...
- 21
2
votes
1
answer
2k
views
Using conditional logistic regression for repeated measures, complete separation (and secondarily, proc logistic)
I'm measuring a single binary outcome, with independent variables:
1) Treatment versus control. Each participant is one or the other.
2) "Before" versus "after" -- each participant has their outcome ...
- 723
2
votes
3
answers
5k
views
GLM high standard errors, but variables are definitely not collinear
When I use a GLM using R, my standard errors are ridiculously high. It can't be because the independent variables are related because they are all distinct ratings for an individual (i.e., interaction ...
- 21
2
votes
1
answer
70
views
SVM - running time for detecting if data is linearly separable?
If my understanding is correct, one way to check if a set of $m$ data points is linearly separable is to use support vector machines to find a maximum margin hyperlane for separating the data; the ...
- 727
-3
votes
1
answer
118
views
what's wrong with my data? [closed]
Sorry,owing to my reputation,I have to delete the above word.
Originally I just want to copy this page's method,the author use titanic data to analyze relationship between fare and survivor.
And I ...
- 11
1
vote
0
answers
79
views
Logistic regression, separating variable (moderator) true in population! [duplicate]
I already checked other posts in this area, but still couldn't get a fit to my issue:
I have the following preconditions:
Software: preferred SPSS v21, possibly R
Sample size: 5655 (will get around ...
- 11
1
vote
1
answer
37
views
Unconnected Linearly Seperable Classification
Consider classifying something like the case shown below (exagerated syntetic example):
If this were a task to classsify into 3 groups, (blue-left, red, blue-right), then a Linear Support Vector ...
- 3,064