# Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

48 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
232 views

### Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect

In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
370 views

### Separation in logistic regression in a complex survey?

Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
403 views

### Weakly-informative priors, complete separation and identifiability in Bayesian logistic regression

Where complete separation may result in non-identifiability of parameter estimates in Bayesian logistic regression, Gelman et al (2008) recommend using weakly-informative priors using a Cauchy ...
926 views

### Handling singular matrix / linear separation in multinomial logit regression

I am doing an analysis of a choice-based conjoint / discrete choice experiment. After using the mlogit.data function in the ...
152 views

### Why PCA is not considered in the taxonomy of blind source separation approaches?

Blind source separation (BSS) approaches are divided in the literature into four methods, including independent component analysis (ICA), sparse component analysis (SCA), and non-negative matrix ...
74 views

### Is A ⊥ B | C where one path active but another inactive?

I'm trying to determine if A ⊥ B | C? I see two paths flowing through elements of C: (1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path) (...
59 views

### How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1k views

### Knowing what exogenous variable cause quasi-separation

I'm trying to build a classifier using a logistic regression and statsmodel is telling me that there is an issue of quasi-separation. Well this isn't an issue! This is exactly what i'm trying to do: ...
1 vote
488 views

### Hauck-Donner effect in ordinal generalized regression

I am running a series of ordinal regressions on a rather large data set (n=3640). With some predictors I had issues with violations of parallel lines assumptions, so I decided to run those with a ...
1 vote
38 views

### Pooling Profile Penalised LTRs in multiple imputation

I am analysizing data from a clinical trial. I used multiple imputation to impute the (binary) outcome variable, which is the only variable with missing data. All of the covariates are categorical and ...
1 vote
27 views

### Cold Start and First Price Auctions

I have the following contrived scenario... I've participated on various auction platforms where I bid on widgets. Assume that win/loss outcomes on individual platforms are well-separated such that for ...
1 vote
63 views

### How many percentage to randomize and how many iterations in a "what-if analysis"?

I've got complete separated data as such: ...
1 vote
196 views

### penalised logistic regression with brglm - warnings

I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
1 vote
31 views

### Separation and Correlation

In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
1 vote
60 views

### What is the *formal* definition of separation in *binary* logistic regression?

I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper https://www.jstor....
1 vote
80 views

### anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
1 vote
43 views

### Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1 vote
34 views

### Neural Networks Mappings( Topology)

Hey I am trying to understand how a neural net performs a kernel trick i.e separate data linearly in high dimensional space. In the example network transforms the input space (2D example) by ...
1 vote
17 views

### any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1 vote
326 views

### When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
1 vote
336 views

### Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1 vote
450 views

### Handling errors: "fitted probabilities numerically 0 or 1 occurred", can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
1 vote
488 views

### A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
1 vote
141 views

### regularized generalized linear model

Since I run into complete separation with logistic regression I try to run a penalized logistic regression for a binomial response variable. It doesn't seem to work for my data. In an example that ...
1 vote
171 views

### How to assess logistic regression & reduce computational effort with imbalanced data

I have a highly imbalanced data set (ratio 1:150) with four predictors, where two are correlated. The data can be found here, you can also see the two figures below. I would like to use logistic ...
1 vote
39 views

### Presenting finite sample examination of asymptotics

From statistical theory, we often obtain results such as $\sqrt n (\theta - \hat \theta) \rightarrow_d N(0, \sigma)$ ie we have a normal limiting distribution. Because this formula says nothing ...
1 vote
358 views

### Why logistic regression functions do not produce the right decision boundary?

I created some data using the following code: ...
1 vote
37 views

### Unconnected Linearly Seperable Classification

Consider classifying something like the case shown below (exagerated syntetic example): If this were a task to classsify into 3 groups, (blue-left, red, blue-right), then a Linear Support Vector ...
1 vote
2k views

### Probit model: marginal effects cannot be estimated because one dummy variable was dropped for predicting failure perfectly

I have a basic question about the -margins- command in Stata: I was wondering if there was a workaround to run marginal effects for a model where one of the dummy ...
1 vote
130 views

### Suggest an exhaustive procedure that will find a separating vector for linearly separable pattern in a finite number of steps

I read in a pattern classification text, that if we consider weight vectors whose components are integer valued, the perceptron procedure would terminate in a finite number of steps. What is the ...
1 vote
821 views

### How does one prove that a separating hyper-plane exists for a linearly separable pattern?

How does one prove that a separating hyper-plane that can be represented as a linear combination of the training samples exists for a linearly separable pattern? Although, it looks pretty obvious ...
1 vote
103 views

### Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
34 views

323 views

### Is this logistf approach the right way to deal with complete separation in my logistic regression?

I am performing lots of association tests between genotypes and binary diseases. The genotypes can be very rare, and my tests often have extreme case control imbalance (e.g. 500 controls for every ...
429 views

### What is the cause of this very high standard error in my logistic regression model?

I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
225 views

### Separating hardly distinguishable classes for easier classification

I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts. I have a dataset with a small number of features ...
500 views

### Cumulative explained variance between scaled and unscaled data

I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
449 views

### Cutting dendrogram at certain point

I have a question about cutting dendrogram like that It shows some hierarchy in the prison I have to cut it to separate the group. Is it possible to cut not all dendrograph at the same height like ...
193 views

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
1k views

### fit warning - matched pairs logistic regression

I'm using matched pairs logistic regression (1-1 matched case-control; Hosmer and Lemeshow 2000) to model differences between vegetation selected at nest sites vs. paired random sites. To do this, I ...
34 views

### Logit model - none of the cases vary on one predictor

What do you do with a categorical predictor (e.g. Black-White-Hispanic) in a logistic model when none of the cases are White, but about half the population studied is? You have to drop this predictor ...
2k views

### Trained Logistic Regression returns 'NAN' for some out of sample data

I'm using MATLAB R2015a, glmfit function for training and glmval for out of sample ...