Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
180
questions
2
votes
1
answer
2k
views
Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation
In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R.
Although I have read many papers on the Firth ...
- 21
1
vote
1
answer
177
views
Quasi/Complete separation due to huge and infinite values
(R statistics)
My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure.
I have a example dataset here
I managed to run ...
4
votes
1
answer
5k
views
If logistic regression is a linear classifier why does it fail on linearly separable data?
Logistic regression is a linear model, decision boundary generated is linear.
If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
- 143
0
votes
0
answers
38
views
Linear separation in higher dimension [duplicate]
I am having a problem comprehending with the relation of kernel, weight and linear separation.
I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$.
And we ...
- 101
0
votes
0
answers
249
views
stratified binary predictor in logistic regression [duplicate]
As part of an assignment, I was working with a dataset containing a breastcancer-related dependent variable (HG) against three independent variables representing ...
- 307
1
vote
0
answers
80
views
anomaly detection : check the separability of normal and abnormal data
I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
- 13
3
votes
1
answer
656
views
Handling quasi-perfect separation in a zero-inflated negative binomial regression in R
I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
- 33
0
votes
1
answer
96
views
Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]
I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
11
votes
1
answer
43k
views
glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric
I've been receiving the warning message in the title and have reviewed posts such as e.g. this one.
I would like to understand how this feature has perfect separation with the target variable, since ...
- 1,538
1
vote
0
answers
43
views
Does separation of data matter in Bayesian power calculations with logistic regression models?
I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
- 209
0
votes
1
answer
375
views
A priori contrast for binomial GLM
after much reading I decided to write because I cannot find a solution to my question.
I already did a priori contrasts before for a continuous variable with normal distribution. Now I have another ...
- 1
1
vote
1
answer
164
views
Is there an upper bound on number of logistic regression responses that yield infinite estimates
Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
- 63
0
votes
0
answers
194
views
Logistic regression with separation and nested design in R
Data:
An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1.
I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
- 93
1
vote
0
answers
34
views
Neural Networks Mappings( Topology)
Hey I am trying to understand how a neural net performs a kernel trick i.e separate data linearly in high dimensional space. In the example network transforms the input space (2D example) by ...
- 772
5
votes
2
answers
481
views
GLMM for count data using square root link in lme4
I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
- 51
0
votes
1
answer
355
views
Kullback-Leibler Divergence as the class separation measure
I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
user223260
1
vote
0
answers
17
views
any problems with Firths Logit model (to deal with separation)
I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
- 209
1
vote
1
answer
577
views
Zero Centering Data
Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
- 141
1
vote
1
answer
1k
views
Can I run a logistic regression with a small sample size and perfect convergence?
I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
- 13
3
votes
1
answer
185
views
R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?
Suppose I have the following data
df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0))
where x = 0 perfectly predicts y = 0. I ...
- 1,514
5
votes
1
answer
2k
views
How to tell which variable is perfectly separated in R [duplicate]
I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
- 249
1
vote
1
answer
610
views
Why do we sum the cost function in a logistic regression?
I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1
vote
1
answer
2k
views
Abnormally high standard error in binary logistic regression
I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
- 13
15
votes
2
answers
20k
views
Issue with complete separation in logistic regression (in R)
I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios.
When estimating the model in R, the ...
- 153
1
vote
0
answers
326
views
When logits are too large in logistic regression, are statistics and p values still interpretable?
I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
- 450
0
votes
1
answer
2k
views
fitted probabilities in logistic regression [closed]
Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
- 151
0
votes
1
answer
129
views
How do I deal with a perfect fitted model?
I have a data set where one independent variable perfectly predicts the outcome.
Specifically, age will perfectly predict if the user will buy the product.
However, I also have a number of other ...
- 3
6
votes
2
answers
815
views
How does perfect separation in logistic regression affect the AUC?
I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
- 449
2
votes
0
answers
59
views
How to calculate a perfect separation in a d dimensional space between n class?
Assuming in a d-dimensional space, we have samples from n class.
The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
- 758
1
vote
1
answer
2k
views
I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message
I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do:
...
- 45
4
votes
2
answers
2k
views
How to identify linearly separable datasets
Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
- 133
1
vote
0
answers
336
views
Does SVM suffer from Quasi-complete separation or perfect separation?
Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
- 11
1
vote
2
answers
103
views
Clustering Separable but Unequally Sized Clusters
I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
0
votes
1
answer
4k
views
Logistic Regression: p values all '1', yet model fits perfectly [duplicate]
I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation.
Here's some economic data. All we have to do is create a model to predict whether the ...
- 29
3
votes
0
answers
403
views
Weakly-informative priors, complete separation and identifiability in Bayesian logistic regression
Where complete separation may result in non-identifiability of parameter estimates in Bayesian logistic regression, Gelman et al (2008) recommend using weakly-informative priors using a Cauchy ...
- 31
4
votes
2
answers
7k
views
Logistic Regression is a Convex Problem but my results show otherwise?
I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
- 846
8
votes
5
answers
3k
views
Why is it desirable to have linear separability in SVM?
Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ?
Can anyone please ...
- 251
1
vote
0
answers
450
views
Handling errors: "fitted probabilities numerically 0 or 1 occurred", can these be ignored/averaged out in a simulation with many trials?
I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
- 2,682
3
votes
1
answer
757
views
How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?
I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
- 171
8
votes
2
answers
2k
views
Is it possible to simulate logistic regression without randomness?
We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
- 35k
1
vote
0
answers
490
views
A mixed effect regression tool based on bayesian priors like bayesglm
I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation.
Unfortunately I could not find an equivalent of empirical ...
- 2,533
4
votes
1
answer
135
views
Using *EDIT* non-informative priors to account for perfect separation in logistic regression?
In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
- 273
7
votes
1
answer
7k
views
Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?
We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings:
Warning messages:
1: glm.fit: algorithm did not ...
- 35k
2
votes
1
answer
316
views
How to use propensity scores that exhibit separation
I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
- 383
15
votes
3
answers
2k
views
Intuition for Support Vector Machines and the hyperplane
In my project I want to create a logistic regression model for predicting binary classification (1 or 0).
I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
- 629
4
votes
3
answers
605
views
Binomial GLM - non-significant difference between 100% opposite groups of observations
What follows is a basic question concerning Binomial GLM's.
Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D -
...
- 242
2
votes
1
answer
5k
views
Multicollinearity and perfect separation in logistic regression: what should I do?
I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
- 21
3
votes
2
answers
947
views
Normally distributed estimated parameters in logistic regression
I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
- 652
3
votes
1
answer
967
views
Does (quasi) complete separation harm prediction in logistic regression?
A client has classified geographical areas into groups. He has done this in various ways, based on substantive concerns, so there is a 2 group division, a 3 group division and so on. He has also ...
- 98.1k
3
votes
1
answer
895
views
Connections between Logistic Regression and Linear Programming
This post Testing for Linear Separability with Linear Programming in R, discusses using linear programming to test if data is linear separable.
What's the connection (if there are any) between LP ...
- 35k