Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

39 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
1k views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
5
votes
0answers
267 views

Separation in logistic regression in a complex survey?

Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
3
votes
0answers
597 views

Handling singular matrix / linear separation in multinomial logit regression

I am doing an analysis of a choice-based conjoint / discrete choice experiment. After using the mlogit.data function in the ...
2
votes
0answers
35 views

likelihood ratio test for logistic regression with a complete separation

Can likelihood ratio test be used for logistic regression in the case of complete separation? I understand that individual parameters can go to infinity so Wald's test is not recommended, but does ...
2
votes
0answers
52 views

Is A ⊥ B | C where one path active but another inactive?

I'm trying to determine if A ⊥ B | C? I see two paths flowing through elements of C: (1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path) (...
2
votes
0answers
52 views

How to calculate a perfect separation in a d dimensional space between n class?

Assuming in a d-dimensional space, we have samples from n class. The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
1
vote
0answers
50 views

How many percentage to randomize and how many iterations in a “what-if analysis”?

I've got complete separated data as such: ...
1
vote
0answers
46 views

penalised logistic regression with brglm - warnings

I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
1
vote
0answers
23 views

Separation and Correlation

In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
1
vote
0answers
55 views

What is the *formal* definition of separation in *binary* logistic regression?

I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper https://www.jstor....
1
vote
0answers
47 views

anomaly detection : check the separability of normal and abnormal data

I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
1
vote
0answers
36 views

Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
0answers
15 views

any problems with Firths Logit model (to deal with separation)

I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1
vote
0answers
145 views

When logits are too large in logistic regression, are statistics and p values still interpretable?

I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
1
vote
0answers
252 views

Does SVM suffer from Quasi-complete separation or perfect separation?

Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
1
vote
0answers
415 views

Handling errors: “fitted probabilities numerically 0 or 1 occurred”, can these be ignored/averaged out in a simulation with many trials?

I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
1
vote
0answers
399 views

A mixed effect regression tool based on bayesian priors like bayesglm

I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation. Unfortunately I could not find an equivalent of empirical ...
1
vote
0answers
135 views

regularized generalized linear model

Since I run into complete separation with logistic regression I try to run a penalized logistic regression for a binomial response variable. It doesn't seem to work for my data. In an example that ...
1
vote
0answers
154 views

How to assess logistic regression & reduce computational effort with imbalanced data

I have a highly imbalanced data set (ratio 1:150) with four predictors, where two are correlated. The data can be found here, you can also see the two figures below. I would like to use logistic ...
1
vote
0answers
36 views

Presenting finite sample examination of asymptotics

From statistical theory, we often obtain results such as $\sqrt n (\theta - \hat \theta) \rightarrow_d N(0, \sigma)$ ie we have a normal limiting distribution. Because this formula says nothing ...
1
vote
0answers
524 views

Knowing what exogenous variable cause quasi-separation

I'm trying to build a classifier using a logistic regression and statsmodel is telling me that there is an issue of quasi-separation. Well this isn't an issue! This is exactly what i'm trying to do: ...
1
vote
0answers
292 views

Why logistic regression functions do not produce the right decision boundary?

I created some data using the following code: ...
1
vote
1answer
31 views

Unconnected Linearly Seperable Classification

Consider classifying something like the case shown below (exagerated syntetic example): If this were a task to classsify into 3 groups, (blue-left, red, blue-right), then a Linear Support Vector ...
1
vote
0answers
1k views

Probit model: marginal effects cannot be estimated because one dummy variable was dropped for predicting failure perfectly

I have a basic question about the -margins- command in Stata: I was wondering if there was a workaround to run marginal effects for a model where one of the dummy ...
1
vote
2answers
92 views

Clustering Separable but Unequally Sized Clusters

I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
0
votes
0answers
27 views

Options when model complexity and separation causes non-convergence in logistic regression

I have created an example data set here My data represent the presence/absence of a particular animal species (data$outcome) and measurements of trees. I would like ...
0
votes
0answers
54 views

How to use the kernel trick on a XOR-like dataset

Let's say that I have the following data: I want to find a transformation of this dataset that will make it linearly separable. My thought was to bring the data around the origin and then multiply $...
0
votes
0answers
8 views

Identify two points from a set as similar

I'm trying to build a model to asses different physiological parameters (Dry weight, yield, Nitrogen concentration, one model for each) from a field based on reflectance from different wavelengths i.e....
0
votes
0answers
130 views

What is the cause of this very high standard error in my logistic regression model?

I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
0
votes
0answers
60 views

1D representation for 2D toy data (about linear separability)

suppose there is a dataset with 2 features x1, and x2. the points (-1;-1); (1; 1); (-3;-3); (4; 4) belong to class 1 and (-1; 1); (1;-1); (-5; 2); (4;-8) belongs to class 2. I am confused in terms of ...
0
votes
0answers
30 views

Separating hardly distinguishable classes for easier classification

I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts. I have a dataset with a small number of features ...
0
votes
0answers
99 views

Cumulative explained variance between scaled and unscaled data

I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
0
votes
0answers
73 views

Cutting dendrogram at certain point

I have a question about cutting dendrogram like that It shows some hierarchy in the prison I have to cut it to separate the group. Is it possible to cut not all dendrograph at the same height like ...
0
votes
0answers
106 views

Logistic regression with separation and nested design in R

Data: An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1. I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
0
votes
0answers
104 views

R logistic fitted probabilities 0 or 1

...
0
votes
0answers
856 views

fit warning - matched pairs logistic regression

I'm using matched pairs logistic regression (1-1 matched case-control; Hosmer and Lemeshow 2000) to model differences between vegetation selected at nest sites vs. paired random sites. To do this, I ...
0
votes
0answers
33 views

Logit model - none of the cases vary on one predictor

What do you do with a categorical predictor (e.g. Black-White-Hispanic) in a logistic model when none of the cases are White, but about half the population studied is? You have to drop this predictor ...
0
votes
0answers
984 views

Trained Logistic Regression returns 'NAN' for some out of sample data

I'm using MATLAB R2015a, glmfit function for training and glmval for out of sample ...
0
votes
0answers
1k views

Using results of Canonical Discriminant Analysis to get overall variable importance?

I have a dataset with thousands of observations pre-assigned to 18 groups and with measures for 8 different variables. I am using canonical discriminant analysis to see how separable my 18 groups are. ...