# Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

137 questions
Filter by
Sorted by
Tagged with
15 views

### Preprocessing on unsupervised learning

I am working on a high dimensional problem that evaluates code readability according to specific metrics. The problem is that there is no 'ground truth' so I need to implement clustering (instead of ...
34 views

### Doubt on d-separation

In the book: Bayesian Networks With Examples in R, the author shows three examples of d-separation: He cites: Then, just a few lines below, the author uses the dsep function, which returns FALSE for ...
47 views

### Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

My question is bolded below, some of my reading notes are also shared for those interested. Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification ...
18 views

### likelihood ratio test for logistic regression with a complete separation

Can likelihood ratio test be used for logistic regression in the case of complete separation? I understand that individual parameters can go to infinity so Wald's test is not recommended, but does ...
35 views

### Clustering Proof of Equation

Greetings! Could anyone enlighten me about the validity of this equation? I'm trying to prove it without success. $K$ is the number of clusters, $C_i$ is the $i$-th cluster, $m_i$ is the number of ...
17 views

### Relationship between Cohesion and Separation

Hello! May someone explain to me please how to prove the equation depiicted with the red line? Thank you in advance!
23 views

### Doesn't separation in data affects Random Forest?

I'd like to know if separation in data points given by a certain variable will affect Random Forest in terms of "being a correct built model"? For example, logistic regression is highly vulnerable in ...
361 views

### Why one result is so wide in this logistic multiple regession

I am doing multiple logistic regression with data with 24 predictor variables and 193 rows. All predictor variables have values of 0 or 1 and outcome variables (OUTVAR) also has only 2 possibilities. ...
36 views

### Is A ⊥ B | C where one path active but another inactive?

I'm trying to determine if A ⊥ B | C? I see two paths flowing through elements of C: (1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path) (...
17 views

### Cutting dendrogram at certain point

I have a question about cutting dendrogram like that It shows some hierarchy in the prison I have to cut it to separate the group. Is it possible to cut not all dendrograph at the same height like ...
52 views

### What is the *formal* definition of separation in *binary* logistic regression?

I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper https://www.jstor....
99 views

### How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
67 views

### Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation

In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R. Although I have read many papers on the Firth ...
47 views

### Quasi/Complete separation due to huge and infinite values

(R statistics) My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure. I have a example dataset here I managed to run ...
178 views

### If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
33 views

### Linear separation in higher dimension [duplicate]

I am having a problem comprehending with the relation of kernel, weight and linear separation. I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$. And we ...
53 views

### Logistic Regression: How to Detect Complete AND Quasi-Separation of Data Points

I have written a logistic regression routine using the Newton-Ralphson algorithm in VBA for use in a class that I teach which uses primarily EXCEL. I want the algorithm to test for complete and quasi-...
7 views

### How to calculate if the given dataset is linearly separable or not using the given discriminant function [duplicate]

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
51 views

### How to manually calculate if the given dataset is linearly separable or not using the given discriminant function

I am student and learning SVM with radial basis kernel function and want to solve the question manually by hand. but I am unable to calculate the values to find out whether the dataset is linearly ...
52 views

### R GLMM: How to deal with separation inflating standard errors in negative binomial regression (count data)

I'm comparing counts of individuals between different localities, and between two seasons (seas_wmo with levels dry and ...
37 views

### anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
99 views

### Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
29 views

### Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]

I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
4k views

### glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
42 views

### Dealing with complete separation in Generalized Additive Models

Related to "How to deal with perfect separation in logistic regression?": I am modelling survival using a binomial mixed model (gamm from ...
32 views

### Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
43 views

### Is there an upper bound on number of logistic regression responses that yield infinite estimates

Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
59 views

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
255 views

### GLMM for count data using square root link in lme4

I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
134 views

### Kullback-Leibler Divergence as the class separation measure

I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
13 views

### any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
148 views

### Zero Centering Data

Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
277 views

### Can I run a logistic regression with a small sample size and perfect convergence?

I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
77 views

### R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?

Suppose I have the following data df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0)) where x = 0 perfectly predicts y = 0. I ...
939 views

### How to tell which variable is perfectly separated in R [duplicate]

I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
442 views

### Why do we sum the cost function in a logistic regression?

I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
702 views

### Abnormally high standard error in binary logistic regression

I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
3k views

### Issue with complete separation in logistic regression (in R)

I am trying to fit a logistic regression model for business defaults. Apart from the dichotomous variable default, the data set includes some performance ratios. When estimating the model in R, the ...
95 views

### When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
2k views

### fitted probabilities in logistic regression [closed]

Is the only reason why fitted probabilities of 0 or 1 occur is that some of your predicting variables(x) are perfect linear combinations of the target(y) variable? Is there any other reason?
118 views

### How do I deal with a perfect fitted model?

I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product. However, I also have a number of other ...
394 views

### How does perfect separation in logistic regression affect the AUC?

I have been working with perfect separation in logistic regression, and I have been assessing models with the AUC statistic. I was wondering what effect perfect separation has on the AUC. My own ...
50 views

### How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1k views

### I think my logistic model is overfitted even with Lasso? R gives me a perfect separation warning message

I have a survey with 25 variables (160 observations for each var). Most are categorical, some are continuous. The variable of interest is categorical (binomial). If I do: ...
846 views

### How to identify linearly separable datasets

Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to ...
188 views

### Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
81 views

### Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
1k views

### Logistic Regression: p values all '1', yet model fits perfectly [duplicate]

I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation. Here's some economic data. All we have to do is create a model to predict whether the ...
I know that logistic regression is a convex problem. Furthermore, from Lemma 1.17 in these optimization lecture notes, if a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is convex, then the ...