Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
48
questions with no upvoted or accepted answers
5
votes
0
answers
232
views
Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect
In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
- 644
5
votes
0
answers
370
views
Separation in logistic regression in a complex survey?
Firth's penalized maximum likelihood estimates, exact logistic regression and Bayesian logistic regression (e.g. bayesglm) can account for separation in logistic regression. But how to account for ...
- 1,566
3
votes
0
answers
403
views
Weakly-informative priors, complete separation and identifiability in Bayesian logistic regression
Where complete separation may result in non-identifiability of parameter estimates in Bayesian logistic regression, Gelman et al (2008) recommend using weakly-informative priors using a Cauchy ...
- 31
3
votes
0
answers
926
views
Handling singular matrix / linear separation in multinomial logit regression
I am doing an analysis of a choice-based conjoint / discrete choice experiment. After using the mlogit.data function in the ...
- 255
2
votes
0
answers
152
views
Why PCA is not considered in the taxonomy of blind source separation approaches?
Blind source separation (BSS) approaches are divided in the literature into four methods, including independent component analysis (ICA), sparse component analysis (SCA), and non-negative matrix ...
- 176
2
votes
0
answers
74
views
Is A ⊥ B | C where one path active but another inactive?
I'm trying to determine if A ⊥ B | C?
I see two paths flowing through elements of C:
(1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path)
(...
- 226
2
votes
0
answers
59
views
How to calculate a perfect separation in a d dimensional space between n class?
Assuming in a d-dimensional space, we have samples from n class.
The best way of separating samples of each class from each other is to have the samples from each class as far as possible from every ...
- 748
2
votes
0
answers
1k
views
Knowing what exogenous variable cause quasi-separation
I'm trying to build a classifier using a logistic regression and statsmodel is telling me that there is an issue of quasi-separation.
Well this isn't an issue! This is exactly what i'm trying to do: ...
- 31
1
vote
0
answers
488
views
Hauck-Donner effect in ordinal generalized regression
I am running a series of ordinal regressions on a rather large data set (n=3640). With some predictors I had issues with violations of parallel lines assumptions, so I decided to run those with a ...
- 11
1
vote
0
answers
38
views
Pooling Profile Penalised LTRs in multiple imputation
I am analysizing data from a clinical trial.
I used multiple imputation to impute the (binary) outcome variable, which is the only variable with missing data.
All of the covariates are categorical and ...
- 11
1
vote
0
answers
27
views
Cold Start and First Price Auctions
I have the following contrived scenario... I've participated on various auction platforms where I bid on widgets. Assume that win/loss outcomes on individual platforms are well-separated such that for ...
- 11
1
vote
0
answers
63
views
How many percentage to randomize and how many iterations in a "what-if analysis"?
I've got complete separated data as such:
...
- 2,071
1
vote
0
answers
196
views
penalised logistic regression with brglm - warnings
I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
- 53
1
vote
0
answers
31
views
Separation and Correlation
In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
- 229
1
vote
0
answers
60
views
What is the *formal* definition of separation in *binary* logistic regression?
I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper
https://www.jstor....
- 11
1
vote
0
answers
80
views
anomaly detection : check the separability of normal and abnormal data
I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
- 13
1
vote
0
answers
43
views
Does separation of data matter in Bayesian power calculations with logistic regression models?
I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
- 209
1
vote
0
answers
34
views
Neural Networks Mappings( Topology)
Hey I am trying to understand how a neural net performs a kernel trick i.e separate data linearly in high dimensional space. In the example network transforms the input space (2D example) by ...
- 772
1
vote
0
answers
17
views
any problems with Firths Logit model (to deal with separation)
I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
- 209
1
vote
0
answers
326
views
When logits are too large in logistic regression, are statistics and p values still interpretable?
I am using glmer() in r to run a mixed logistic regression with 3 categorical (dichotomous) predictors. The outcome measure is whether or not a participant responded correctly to a memory check. This ...
- 450
1
vote
0
answers
336
views
Does SVM suffer from Quasi-complete separation or perfect separation?
Are SVMs, which use the hinge loss, suffer from Quasi-complete separation or perfect separation? Why or why not? Could you please give mathematical justification?
- 11
1
vote
0
answers
450
views
Handling errors: "fitted probabilities numerically 0 or 1 occurred", can these be ignored/averaged out in a simulation with many trials?
I have been getting the following errors in R because of trying to fit propensity scores that are well-stratified. I am currently looping over a model simulation ...
- 2,672
1
vote
0
answers
488
views
A mixed effect regression tool based on bayesian priors like bayesglm
I use extensively Gelman's bayesglm for the every day use due to the great stability of the estimates especially in the case of separation.
Unfortunately I could not find an equivalent of empirical ...
- 2,483
1
vote
0
answers
141
views
regularized generalized linear model
Since I run into complete separation with logistic regression I try to run a penalized logistic regression for a binomial response variable.
It doesn't seem to work for my data. In an example that ...
- 343
1
vote
0
answers
171
views
How to assess logistic regression & reduce computational effort with imbalanced data
I have a highly imbalanced data set (ratio 1:150) with four predictors, where two are correlated. The data can be found here, you can also see the two figures below.
I would like to use logistic ...
- 343
1
vote
0
answers
39
views
Presenting finite sample examination of asymptotics
From statistical theory, we often obtain results such as
$\sqrt n (\theta - \hat \theta) \rightarrow_d N(0, \sigma)$
ie we have a normal limiting distribution.
Because this formula says nothing ...
- 19.3k
1
vote
0
answers
358
views
Why logistic regression functions do not produce the right decision boundary?
I created some data using the following code:
...
- 627
1
vote
1
answer
37
views
Unconnected Linearly Seperable Classification
Consider classifying something like the case shown below (exagerated syntetic example):
If this were a task to classsify into 3 groups, (blue-left, red, blue-right), then a Linear Support Vector ...
- 3,064
1
vote
0
answers
2k
views
Probit model: marginal effects cannot be estimated because one dummy variable was dropped for predicting failure perfectly
I have a basic question about the -margins- command in Stata: I was wondering if there was a workaround to run marginal effects for a model where one of the dummy ...
- 1,715
1
vote
0
answers
130
views
Suggest an exhaustive procedure that will find a separating vector for linearly separable pattern in a finite number of steps
I read in a pattern classification text, that if we consider weight vectors whose components are integer valued, the perceptron procedure would terminate in a finite number of steps.
What is the ...
- 487
1
vote
0
answers
821
views
How does one prove that a separating hyper-plane exists for a linearly separable pattern?
How does one prove that a separating hyper-plane that can be represented as a linear combination of the training samples exists for a linearly separable pattern?
Although, it looks pretty obvious ...
- 487
1
vote
2
answers
103
views
Clustering Separable but Unequally Sized Clusters
I am trying to cluster the data shown below. The clusters are clearly separable. I've tried k-means and EM clustering (Gaussian mixture), however, both techniques divide the large main cluster into ...
0
votes
0
answers
34
views
Non-optimal separating hyperplane?
This is exercise 4.7 from Elements of Statistical Learning by Hastie, Tibshirani and Friedman.
Consider the generalisation of the perceptron learning algorithm with the goal of minimising $D^*(\beta, \...
- 165
0
votes
0
answers
41
views
High frequency zero-inflated negative binomial model with Hauck-Donner effect
I have high-frequency (daily) data with overdispersion and a high amount of zeros. I know that a zero-inflated negative binomial is my best option for a model. I am using the library ...
- 11
0
votes
0
answers
48
views
On the use of the sampling year as fixed-effect
I have my database in temporal blocks (so I have my occurrence sites into the year of sampling of a few years with maximum 7 years). In order to fix the complete separation problem that I also have, ...
- 155
0
votes
1
answer
66
views
Is there any impact on the results of random forest if we have perfect separation in the database?
I'm having a perfect separation in my database. One of the solutions is to switch randomly some of the values of the dependent variable to the opposite for the variables that cause the problem this ...
- 155
0
votes
0
answers
93
views
Options when model complexity and separation causes non-convergence in logistic regression
I have created an example data set here
My data represent the presence/absence of a particular animal species (data$outcome) and measurements of trees. I would like ...
- 799
0
votes
0
answers
323
views
Is this logistf approach the right way to deal with complete separation in my logistic regression?
I am performing lots of association tests between genotypes and binary diseases. The genotypes can be very rare, and my tests often have extreme case control imbalance (e.g. 500 controls for every ...
- 143
0
votes
0
answers
429
views
What is the cause of this very high standard error in my logistic regression model?
I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
- 1
0
votes
0
answers
225
views
Separating hardly distinguishable classes for easier classification
I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts.
I have a dataset with a small number of features ...
- 111
0
votes
0
answers
500
views
Cumulative explained variance between scaled and unscaled data
I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
- 111
0
votes
0
answers
449
views
Cutting dendrogram at certain point
I have a question about cutting dendrogram like that
It shows some hierarchy in the prison
I have to cut it to separate the group.
Is it possible to cut not all dendrograph at the same height like ...
- 1
0
votes
0
answers
193
views
Logistic regression with separation and nested design in R
Data:
An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1.
I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
- 93
0
votes
0
answers
1k
views
fit warning - matched pairs logistic regression
I'm using matched pairs logistic regression (1-1 matched case-control; Hosmer and Lemeshow 2000) to model differences between vegetation selected at nest sites vs. paired random sites. To do this, I ...
- 235
0
votes
0
answers
34
views
Logit model - none of the cases vary on one predictor
What do you do with a categorical predictor (e.g. Black-White-Hispanic) in a logistic model when none of the cases are White, but about half the population studied is? You have to drop this predictor ...
- 1
0
votes
0
answers
2k
views
Trained Logistic Regression returns 'NAN' for some out of sample data
I'm using MATLAB R2015a, glmfit function for training and glmval for out of sample ...
- 3,911
0
votes
0
answers
1k
views
Using results of Canonical Discriminant Analysis to get overall variable importance?
I have a dataset with thousands of observations pre-assigned to 18 groups and with measures for 8 different variables. I am using canonical discriminant analysis to see how separable my 18 groups are. ...
- 265
0
votes
0
answers
440
views
Why the analysis of deviance of glm gives df=0 for one covariate?
I am applying a logistic regression on the effect of dose, age, PS, menopausal and pairID on the response variable. The data come from a case-control study where controls were matched/paired based on ...
- 2,011