Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

0
votes
1answer
6 views

Hyperplanes for clustering [on hold]

So I was wondering if hyperplanes could be used to cluster unlabeled data. In particular, can we use the signed distance of a point to a hyperplane to construct the clusters? (so points with a ...
0
votes
0answers
6 views

How to calculate if the given dataset is linearly separable or not using the given discriminant function [duplicate]

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
16 views

How to manually calculate if the given dataset is linearly separable or not using the given discriminant function

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
0
votes
0answers
22 views

R GLMM: How to deal with separation inflating standard errors in negative binomial regression (count data)

I'm comparing counts of individuals between different localities, and between two seasons (seas_wmo with levels dry and ...
1
vote
0answers
27 views

anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
0
votes
0answers
13 views

Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
0
votes
1answer
18 views

Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]

I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
2
votes
1answer
60 views

glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
0
votes
0answers
17 views

Dealing with complete separation in Generalized Additive Models

Related to "How to deal with perfect separation in logistic regression?": I am modelling survival using a binomial mixed model (gamm from ...
1
vote
0answers
17 views

Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
1answer
27 views

Is there an upper bound on number of logistic regression responses that yield infinite estimates

Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
0
votes
0answers
40 views

Logistic regression with separation and nested design in R

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
5
votes
2answers
220 views

GLMM for count data using square root link in lme4

I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
0
votes
1answer
64 views

Kullback-Leibler Divergence as the class separation measure

I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
0
votes
0answers
44 views

other causes of perfect separation

I have a dataset of 300,000 observations. Strangely enough when I use logit I get the following error: ...
0
votes
0answers
17 views

how to report marginal effects of variables omitted due to perfect separation?

i have a few variables that come up as omitted in a probit model but are necessary for a later heckman model, when i write up my results what is the custom for reporting the missing coefficients in ...
1
vote
0answers
11 views

any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1
vote
1answer
52 views

Zero Centering Data

Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
1
vote
1answer
101 views

Can I run a logistic regression with a small sample size and perfect convergence?

I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
2
votes
1answer
68 views

R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?

Suppose I have the following data df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0)) where x = 0 perfectly predicts y = 0. I ...
3
votes
1answer
448 views

How to tell which variable is perfectly separated in R [duplicate]

I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
1
vote
1answer
398 views

Why do we sum the cost function in a logistic regression?

I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1
vote
1answer
361 views

Abnormally high standard error in binary logistic regression

I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
4
votes
1answer
1k views

Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
1
vote
0answers
83 views

When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
0
votes
1answer
1k views

fitted probabilities in logistic regression [closed]

Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
0
votes
1answer
116 views

How do I deal with a perfect fitted model?

I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product. However, I also have a number of other ...
5
votes
2answers
251 views

How does perfect separation in logistic regression affect the AUC?

I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
2
votes
0answers
46 views

How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1
vote
1answer
1k views

I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
0
votes
0answers
270 views

error: GEE function in R “logistic model for probability has fitted value very close to 1” [duplicate]

I am trying to fit a GEE model for repeat measure outcome (binary). But R gave me the following error information: ...
3
votes
2answers
508 views

How to identify linearly separable datasets

Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
1
vote
0answers
134 views

Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1
vote
2answers
63 views

Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
-1
votes
1answer
731 views

Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
3
votes
2answers
2k views

Logistic Regression is a Convex Problem but my results show otherwise?

I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...
8
votes
4answers
1k views

Why is it desirable to have linear separability in SVM?

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ? Can anyone please ...
1
vote
0answers
319 views

Handling errors: “fitted probabilities numerically 0 or 1 occurred”, can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
3
votes
1answer
267 views

How can a logistic regression model have predictors that are highly insignificant and still perform excellently in the test dataset?

I have built a classification model using a highly imbalanced dataset to be found in the ROSE package of R called hacide, containing 1,000 observations of which only 2% are positive. My model ...
8
votes
2answers
1k views

Is it possible to simulate logistic regression without randomness?

We can simulate linear regression without randomness, which means we make $y=X\beta$ instead of $y=X\beta+\epsilon$. Then if we fit a linear model the coefficients will be identical to the "ground ...
1
vote
0answers
240 views

A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
4
votes
1answer
73 views

Using *EDIT* non-informative priors to account for perfect separation in logistic regression?

In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation ...
7
votes
1answer
2k views

Is it possible to get fitted values 0 or 1 in logistic regression when the fitting algorithm converges?

We have comprehensive coverage on the topic of perfect separation in logistic regression. When it happens in R we usually see two warnings: Warning messages: 1: glm.fit: algorithm did not ...
1
vote
1answer
73 views

How to use propensity scores that exhibit separation

I am using probability scoring on a data set where one variable has very small area of common support between treated and control (a small area in the middle exists). When I run a logit regression in ...
14
votes
3answers
2k views

Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
0
votes
0answers
52 views

Logistic regression (inf) [duplicate]

While doing uni-variable analysis in logistic regression where the outcome is binary and the independent variable is continuous, I got these results for two of my variables with very big SE. Can you ...
3
votes
2answers
112 views

Binomial GLM - non-significant difference between 100% opposite groups of observations

What follows is a basic question concerning Binomial GLM's. Suppose we have a set of observations where a binary response was measured in three different treatments, A, C and D - ...
3
votes
1answer
2k views

Multicollinearity and perfect separation in logistic regression: what should I do?

I have a dataset composed of 61 variables a qualitative one y=(0 or 1) and 60 other quantitative variables and 40000 observations. I want to do logistic regression,...
3
votes
2answers
203 views

Normally distributed estimated parameters in logistic regression

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally ...
4
votes
1answer
430 views

Does (quasi) complete separation harm prediction in logistic regression?

A client has classified geographical areas into groups. He has done this in various ways, based on substantive concerns, so there is a 2 group division, a 3 group division and so on. He has also ...