# Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

180 questions
Filter by
Sorted by
Tagged with
2k views

### Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation

In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R. Although I have read many papers on the Firth ...
• 21
1 vote
177 views

### Quasi/Complete separation due to huge and infinite values

(R statistics) My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure. I have a example dataset here I managed to run ...
5k views

### If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
• 143
38 views

### Linear separation in higher dimension [duplicate]

I am having a problem comprehending with the relation of kernel, weight and linear separation. I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$. And we ...
249 views

### stratified binary predictor in logistic regression [duplicate]

As part of an assignment, I was working with a dataset containing a breastcancer-related dependent variable (HG) against three independent variables representing ...
• 307
1 vote
80 views

### anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
• 13
656 views

### Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
96 views

### Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]

I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
43k views

### glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
• 1,538
1 vote
43 views

### Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
• 209
375 views

### A priori contrast for binomial GLM

after much reading I decided to write because I cannot find a solution to my question. I already did a priori contrasts before for a continuous variable with normal distribution. Now I have another ...
• 1
1 vote
164 views

### Is there an upper bound on number of logistic regression responses that yield infinite estimates

Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
• 63
194 views

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
• 93
1 vote
34 views

### Neural Networks Mappings( Topology)

Hey I am trying to understand how a neural net performs a kernel trick i.e separate data linearly in high dimensional space. In the example network transforms the input space (2D example) by ...
• 772
481 views

### GLMM for count data using square root link in lme4

I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
• 51
355 views

### Kullback-Leibler Divergence as the class separation measure

I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
1 vote
17 views

### any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
• 209
1 vote
577 views

### Zero Centering Data

Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
• 141
1 vote
1k views

### Can I run a logistic regression with a small sample size and perfect convergence?

I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
• 13
185 views

### R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?

Suppose I have the following data df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0)) where x = 0 perfectly predicts y = 0. I ...
• 1,514
2k views

### How to tell which variable is perfectly separated in R [duplicate]

I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
• 249
1 vote
610 views

### Why do we sum the cost function in a logistic regression?

I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1 vote
2k views

### Abnormally high standard error in binary logistic regression

I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
20k views

### Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
• 153
1 vote
326 views

### When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
• 450
2k views

### fitted probabilities in logistic regression [closed]

Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
• 151
129 views

### How do I deal with a perfect fitted model?

I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product. However, I also have a number of other ...
815 views

### How does perfect separation in logistic regression affect the AUC?

I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
• 449
59 views

### How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
• 758
1 vote
2k views

### I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
2k views

### How to identify linearly separable datasets

Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
• 133
1 vote
336 views

### Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
• 11
1 vote
103 views

### Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
4k views

### Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
• 29
403 views

### Weakly-informative priors, complete separation and identifiability in Bayesian logistic regression

Where complete separation may result in non-identifiability of parameter estimates in Bayesian logistic regression, Gelman et al (2008) recommend using weakly-informative priors using a Cauchy ...
• 31
7k views

### Logistic Regression is a Convex Problem but my results show otherwise?

I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
• 846
3k views

### Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
• 251
1 vote
450 views

### Handling errors: "fitted probabilities numerically 0 or 1 occurred", can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
• 2,682
757 views

### How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
• 171
2k views

### Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
• 35k
1 vote
490 views

### A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
• 2,533
135 views

### Using *EDIT* non-informative priors to account for perfect separation in logistic regression?

In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
• 273
7k views

### Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
• 35k
316 views

### How to use propensity scores that exhibit separation

I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
• 383
2k views

### Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
• 629
605 views

### Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
5k views

### Multicollinearity and perfect separation in logistic regression: what should I do?

I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
• 21
947 views

### Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
• 652
967 views

### Does (quasi) complete separation harm prediction in logistic regression?

A client has classified geographical areas into groups. He has done this in various ways, based on substantive concerns, so there is a 2 group division, a 3 group division and so on. He has also ...
• 98.1k