Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

Filter by
Sorted by
Tagged with
1
vote
0answers
47 views

What is the *formal* definition of separation in *binary* logistic regression?

I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper https://www.jstor....
0
votes
0answers
27 views

How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
1
vote
1answer
31 views

Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation

In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R. Although I have read many papers on the Firth ...
0
votes
1answer
39 views

Quasi/Complete separation due to huge and infinite values

(R statistics) My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure. I have a example dataset here I managed to run ...
0
votes
0answers
33 views

Linear separation in higher dimension [duplicate]

I am having a problem comprehending with the relation of kernel, weight and linear separation. I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$. And we ...
0
votes
0answers
33 views

Logistic Regression: How to Detect Complete AND Quasi-Separation of Data Points

I have written a logistic regression routine using the Newton-Ralphson algorithm in VBA for use in a class that I teach which uses primarily EXCEL. I want the algorithm to test for complete and quasi-...
0
votes
0answers
7 views

How to calculate if the given dataset is linearly separable or not using the given discriminant function [duplicate]

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
48 views

How to manually calculate if the given dataset is linearly separable or not using the given discriminant function

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
39 views

R GLMM: How to deal with separation inflating standard errors in negative binomial regression (count data)

I'm comparing counts of individuals between different localities, and between two seasons (seas_wmo with levels dry and ...
1
vote
0answers
35 views

anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
2
votes
1answer
75 views

Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
0
votes
1answer
22 views

Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]

I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
2
votes
1answer
2k views

glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
0
votes
0answers
35 views

Dealing with complete separation in Generalized Additive Models

Related to "How to deal with perfect separation in logistic regression?": I am modelling survival using a binomial mixed model (gamm from ...
1
vote
0answers
29 views

Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
1answer
39 views

Is there an upper bound on number of logistic regression responses that yield infinite estimates

Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
0
votes
0answers
52 views

Logistic regression with separation and nested design in R

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
5
votes
2answers
242 views

GLMM for count data using square root link in lme4

I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
0
votes
1answer
102 views

Kullback-Leibler Divergence as the class separation measure

I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
1
vote
0answers
13 views

any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1
vote
1answer
88 views

Zero Centering Data

Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
1
vote
1answer
154 views

Can I run a logistic regression with a small sample size and perfect convergence?

I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
2
votes
1answer
72 views

R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?

Suppose I have the following data df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0)) where x = 0 perfectly predicts y = 0. I ...
4
votes
1answer
711 views

How to tell which variable is perfectly separated in R [duplicate]

I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
1
vote
1answer
434 views

Why do we sum the cost function in a logistic regression?

I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1
vote
1answer
552 views

Abnormally high standard error in binary logistic regression

I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
4
votes
1answer
2k views

Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
1
vote
0answers
93 views

When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
0
votes
1answer
2k views

fitted probabilities in logistic regression [closed]

Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
0
votes
1answer
117 views

How do I deal with a perfect fitted model?

I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product. However, I also have a number of other ...
5
votes
2answers
323 views

How does perfect separation in logistic regression affect the AUC?

I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
2
votes
0answers
48 views

How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1
vote
1answer
1k views

I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
3
votes
2answers
703 views

How to identify linearly separable datasets

Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
1
vote
0answers
162 views

Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1
vote
2answers
74 views

Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
-1
votes
1answer
966 views

Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
4
votes
2answers
3k views

Logistic Regression is a Convex Problem but my results show otherwise?

I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
8
votes
4answers
1k views

Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
1
vote
0answers
359 views

Handling errors: “fitted probabilities numerically 0 or 1 occurred”, can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
3
votes
1answer
352 views

How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
8
votes
2answers
1k views

Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
1
vote
0answers
277 views

A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
4
votes
1answer
79 views

Using *EDIT* non-informative priors to account for perfect separation in logistic regression?

In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
6
votes
1answer
3k views

Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
1
vote
1answer
93 views

How to use propensity scores that exhibit separation

I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
14
votes
3answers
2k views

Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
3
votes
2answers
140 views

Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
2
votes
1answer
3k views

Multicollinearity and perfect separation in logistic regression: what should I do?

I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
3
votes
2answers
229 views

Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...