Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

Filter by
Sorted by
Tagged with
12
votes
1answer
10k views

Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
1
vote
0answers
145 views

When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
0
votes
1answer
2k views

fitted probabilities in logistic regression [closed]

Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
0
votes
1answer
121 views

How do I deal with a perfect fitted model?

I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product. However, I also have a number of other ...
6
votes
2answers
536 views

How does perfect separation in logistic regression affect the AUC?

I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
2
votes
0answers
52 views

How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1
vote
1answer
2k views

I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
3
votes
2answers
1k views

How to identify linearly separable datasets

Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
1
vote
0answers
252 views

Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1
vote
2answers
92 views

Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
0
votes
1answer
2k views

Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
4
votes
2answers
5k views

Logistic Regression is a Convex Problem but my results show otherwise?

I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
8
votes
4answers
2k views

Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
1
vote
0answers
415 views

Handling errors: “fitted probabilities numerically 0 or 1 occurred”, can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
3
votes
1answer
557 views

How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
8
votes
2answers
2k views

Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
1
vote
0answers
399 views

A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
4
votes
1answer
92 views

Using *EDIT* non-informative priors to account for perfect separation in logistic regression?

In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
7
votes
1answer
5k views

Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
1
vote
1answer
193 views

How to use propensity scores that exhibit separation

I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
15
votes
3answers
2k views

Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
3
votes
2answers
216 views

Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
2
votes
1answer
5k views

Multicollinearity and perfect separation in logistic regression: what should I do?

I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
3
votes
2answers
430 views

Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
3
votes
1answer
644 views

Does (quasi) complete separation harm prediction in logistic regression?

A client has classified geographical areas into groups. He has done this in various ways, based on substantive concerns, so there is a 2 group division, a 3 group division and so on. He has also ...
3
votes
1answer
674 views

Connections between Logistic Regression and Linear Programming

This post Testing for Linear Separability with Linear Programming in R, discusses using linear programming to test if data is linear separable. What's the connection (if there are any) between LP ...
2
votes
1answer
227 views

Complete separation in logistic regression with only one direction

In this paper Dealing with Separation in Logistic Regression Models some various types of complete separation are discussed: direction of the separation is positive if and only if $s_i = 1 \...
36
votes
4answers
14k views

Why does logistic regression become unstable when classes are well-separated?

Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
0
votes
1answer
979 views

perfect separation logistic regression [duplicate]

in continuity to the post stepwise logistic regression non significative variables(high p-values) and as demanded by matthew this is a post explaining the data i have and the problem in fact i have a ...
1
vote
1answer
285 views

Are neural networks able to separate these classes?

I know that MLP can separate classes even if the decision boundary is nonlinear. However, in all cases that I've seen the boundary was just a single 'unclosed' line. Is the MLP able to solve the ...
8
votes
5answers
5k views

When does logistic regression not work properly?

I need to find a situation in which logistic regression does not work well. Furthermore, I would like to know when a random forest might perform better than a logistic regression model.
2
votes
0answers
677 views

What's the reason for large beta coefficient standard error and estimate in binary logistic regression? [closed]

I'm using SPSS for binary logistic regression. The result showed that one of the variables has a large beta coefficient and standard error. Here are some results: Can some explain to me what is the ...
0
votes
1answer
47 views

How to pick two features that can linearly separate two classes as much as possible

I have a few dataset, each has about 120 features/dimensions and two classes (e.g. A and B) in it. Now I'd like to visualize the dataset from just two dimensions without doing any dimensionality ...
0
votes
0answers
104 views

R logistic fitted probabilities 0 or 1

...
1
vote
0answers
56 views

Logistic plot seems wrong [duplicate]

I'm a newbie to R and am hoping someone can show me the error of my ways. I'm getting what appears to be a logistic regression and plot but I wasn't expecting the slope around 25.2 to be so steep and ...
2
votes
1answer
153 views

Will the p value become useless in such case: logistic regression with perfect separation?

I think I understand the perfect separation problem in logistic regression and answered my own question in this post from optimization perspective. Is there any intuitive explanation of why logistic ...
23
votes
1answer
6k views

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
13
votes
2answers
3k views

Is R's glm function useless in a big data / machine learning setting? [duplicate]

I am surprised that R’s glm will “break” (not converge with default setting) for the following “toy” example (binary classification with ~50k data, ~10 features), ...
8
votes
1answer
760 views

How does Python Scikit Learn handle linear separation problem in logistic regression?

There are already posts about warnings from R dealing with logistic regression and linear separation such as this one. I just wanna make sure if in Python Scikit Learn this problem is all solved by ...
1
vote
1answer
49 views

Which test should I use? Hypothesis: Porcupines prefer certain plant genotypes over others in a common garden

Hypothesis: Porcupines prefer certain plant genotypes over others in a common garden. We visually inspected just over 2,500 trees in a common garden for evidence of herbivory on woody tissues. Each ...
2
votes
2answers
1k views

Variable selection for logistic regression with separated data

I have a fairly large dataset ($\approx 3 \bar{M}$ observations for a dozen candidate predictors) and I would like to perform a logistic regression on that dataset. I have a problem of separation in ...
3
votes
1answer
601 views

Quasi-separation when running Cox regression comparing 3 risk groups?

This is my first post here, so please bear with me! I'm comparing several biomarkers with Kaplan-Meier curves and calculating hazard ratios for different risk groups (defined by a certain, well ...
1
vote
1answer
283 views

How can I prove if 2 datasets are separable or not?

I want to create a classification model, derived from real spectral measurements. My data is in the form of samples x features. The number of features is 424 and the number of samples is in the order ...
1
vote
0answers
135 views

regularized generalized linear model

Since I run into complete separation with logistic regression I try to run a penalized logistic regression for a binomial response variable. It doesn't seem to work for my data. In an example that ...
1
vote
1answer
109 views

Can univariate analysis find the p value, HZ ratio from the 100% alive group analysis

I try to compare two groups in univariate analysis but there are some problems because one group was alive only (no dead)
21
votes
1answer
17k views

Understanding complete separation for logistic regression [duplicate]

Why does logistic regression not converge for a linearly separable data set? For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
1
vote
0answers
154 views

How to assess logistic regression & reduce computational effort with imbalanced data

I have a highly imbalanced data set (ratio 1:150) with four predictors, where two are correlated. The data can be found here, you can also see the two figures below. I would like to use logistic ...
2
votes
1answer
148 views

Really weird results for a logistic model - is it due to high frequency of one value on response variable?

I am trying to test whether experimental group (a vs b) influences the probability of some binary outcome, but the model results are strange. The code I'm using: ...
1
vote
0answers
36 views

Presenting finite sample examination of asymptotics

From statistical theory, we often obtain results such as $\sqrt n (\theta - \hat \theta) \rightarrow_d N(0, \sigma)$ ie we have a normal limiting distribution. Because this formula says nothing ...
6
votes
3answers
1k views

Strange outcomes in binary logistic regression in SPSS

I did a binary logistic regression with SPSS 23 and I found some strange outcomes. This is for NOACprev until No_Prev_treatment, the last 6 variables. First of all they have very high outcomes for B, ...