Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
125
questions
1
vote
0answers
47 views
What is the *formal* definition of separation in *binary* logistic regression?
I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper
https://www.jstor....
0
votes
0answers
27 views
How to show or prove a dataset is not linearly separable
I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
1
vote
1answer
31 views
Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation
In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R.
Although I have read many papers on the Firth ...
0
votes
1answer
39 views
Quasi/Complete separation due to huge and infinite values
(R statistics)
My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure.
I have a example dataset here
I managed to run ...
0
votes
0answers
33 views
Linear separation in higher dimension [duplicate]
I am having a problem comprehending with the relation of kernel, weight and linear separation.
I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$.
And we ...
0
votes
0answers
33 views
Logistic Regression: How to Detect Complete AND Quasi-Separation of Data Points
I have written a logistic regression routine using the Newton-Ralphson algorithm in VBA for use in a class that I teach which uses primarily EXCEL. I want the algorithm to test for complete and quasi-...
0
votes
0answers
7 views
How to calculate if the given dataset is linearly separable or not using the given discriminant function [duplicate]
I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
48 views
How to manually calculate if the given dataset is linearly separable or not using the given discriminant function
I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
39 views
R GLMM: How to deal with separation inflating standard errors in negative binomial regression (count data)
I'm comparing counts of individuals between different localities, and between two seasons (seas_wmo with levels dry and ...
1
vote
0answers
35 views
anomaly detection : check the separability of normal and abnormal data
I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
2
votes
1answer
75 views
Handling quasi-perfect separation in a zero-inflated negative binomial regression in R
I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
0
votes
1answer
22 views
Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]
I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
2
votes
1answer
2k views
glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric
I've been receiving the warning message in the title and have reviewed posts such as e.g. this one.
I would like to understand how this feature has perfect separation with the target variable, since ...
0
votes
0answers
35 views
Dealing with complete separation in Generalized Additive Models
Related to "How to deal with perfect separation in logistic regression?":
I am modelling survival using a binomial mixed model (gamm from ...
1
vote
0answers
29 views
Does separation of data matter in Bayesian power calculations with logistic regression models?
I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
1answer
39 views
Is there an upper bound on number of logistic regression responses that yield infinite estimates
Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
0
votes
0answers
52 views
Logistic regression with separation and nested design in R
Data:
An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1.
I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
5
votes
2answers
242 views
GLMM for count data using square root link in lme4
I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
0
votes
1answer
102 views
Kullback-Leibler Divergence as the class separation measure
I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
1
vote
0answers
13 views
any problems with Firths Logit model (to deal with separation)
I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1
vote
1answer
88 views
Zero Centering Data
Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
1
vote
1answer
154 views
Can I run a logistic regression with a small sample size and perfect convergence?
I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
2
votes
1answer
72 views
R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?
Suppose I have the following data
df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0))
where x = 0 perfectly predicts y = 0. I ...
4
votes
1answer
711 views
How to tell which variable is perfectly separated in R [duplicate]
I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
1
vote
1answer
434 views
Why do we sum the cost function in a logistic regression?
I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1
vote
1answer
552 views
Abnormally high standard error in binary logistic regression
I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
4
votes
1answer
2k views
Issue with complete separation in logistic regression (in R)
I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios.
When estimating the model in R, the ...
1
vote
0answers
93 views
When logits are too large in logistic regression, are statistics and p values still interpretable?
I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
0
votes
1answer
2k views
fitted probabilities in logistic regression [closed]
Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
0
votes
1answer
117 views
How do I deal with a perfect fitted model?
I have a data set where one independent variable perfectly predicts the outcome.
Specifically, age will perfectly predict if the user will buy the product.
However, I also have a number of other ...
5
votes
2answers
323 views
How does perfect separation in logistic regression affect the AUC?
I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
2
votes
0answers
48 views
How to calculate a perfect separation in a d dimensional space between n class?
Assuming in a d-dimensional space, we have samples from n class.
The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1
vote
1answer
1k views
I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message
I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do:
...
3
votes
2answers
703 views
How to identify linearly separable datasets
Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
1
vote
0answers
162 views
Does SVM suffer from Quasi-complete separation or perfect separation?
Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1
vote
2answers
74 views
Clustering Separable but Unequally Sized Clusters
I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
-1
votes
1answer
966 views
Logistic Regression: p values all '1', yet model fits perfectly [duplicate]
I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation.
Here's some economic data. All we have to do is create a model to predict whether the ...
4
votes
2answers
3k views
Logistic Regression is a Convex Problem but my results show otherwise?
I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
8
votes
4answers
1k views
Why is it desirable to have linear separability in SVM?
Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ?
Can anyone please ...
1
vote
0answers
359 views
Handling errors: “fitted probabilities numerically 0 or 1 occurred”, can these be ignored/averaged out in a simulation with many trials?
I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
3
votes
1answer
352 views
How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?
I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
8
votes
2answers
1k views
Is it possible to simulate logistic regression without randomness?
We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
1
vote
0answers
277 views
A mixed effect regression tool based on bayesian priors like bayesglm
I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation.
Unfortunately I could not find an equivalent of empirical ...
4
votes
1answer
79 views
Using *EDIT* non-informative priors to account for perfect separation in logistic regression?
In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
6
votes
1answer
3k views
Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?
We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings:
Warning messages:
1: glm.fit: algorithm did not ...
1
vote
1answer
93 views
How to use propensity scores that exhibit separation
I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
14
votes
3answers
2k views
Intuition for Support Vector Machines and the hyperplane
In my project I want to create a logistic regression model for predicting binary classification (1 or 0).
I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
3
votes
2answers
140 views
Binomial GLM - non-significant difference between 100% opposite groups of observations
What follows is a basic question concerning Binomial GLM's.
Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D -
...
2
votes
1answer
3k views
Multicollinearity and perfect separation in logistic regression: what should I do?
I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
3
votes
2answers
229 views
Normally distributed estimated parameters in logistic regression
I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
