Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
187
questions
2
votes
1
answer
71
views
What is the difference between using logistf and brglm2 when dealing with complete separation in a logistic regression?
I am trying to looking at how the three factors A (5 levels, a-e), B (2 levels, a and b) and C (2 levels, a and b) affect the likelihood of event Y (1 = occured, 0 = did not occur). I initially ran a ...
- 441
1
vote
0
answers
36
views
How to show that MLE of probit regression does not exist due to data separability
Claim
The claim is the is the following:
Assume we have the simple probit model $E(y_i|x_i ) = Φ(α+\beta x_i)$. Now suppose that $y_i = 1$ for all $x_i ≤ 10$ and $y_i = 0$ for all $x_i > 10$. Then $...
- 623
2
votes
1
answer
72
views
Fisher Scoring Algorithm did not converge however model output looks fine in R
I am currently trying to fit a logistic regression model in R with separated data to analyse the problems occuring in such a case. Indeed for the following model warning messages occured, however in ...
- 23
1
vote
0
answers
44
views
emmeans differences in logistic regression give wrong significance in case of complete separation
Suppose (fictitious example) we have a logistic model for dependent "Happy" (0/1), and factor "Age" ("young", "middle", "old") as independent variable ...
- 1,818
0
votes
0
answers
51
views
Generalized mixed effect logistic regression model and strange p values (maybe separation of data)? [duplicate]
I'm running a mixed effect logistic regression model (function glmer from the package lme4) in RStudio with two random ...
- 175
1
vote
2
answers
119
views
GLMER and p values when all outputs are 1: strange results?
I'm running a mixed effect logistic regression model (function glmer from the package lme4) in RStudio with two random ...
- 175
0
votes
0
answers
27
views
Is it possible to run a zero-inflated negative binomial model with complete separation? [duplicate]
I am trying to analyze how the number of events Y is influenced by three factors A (4 levels), B (2 levels) and C (2 levels) and the interactions between the three.
Initially trying a poisson ...
- 441
0
votes
0
answers
247
views
Warning: Hauck-Donner effect detected in the following estimate(s): INTERCEPT
This is a quick question because I could not find an answer online.
I ran a truncated Poisson model and I got the warning in the title (perfect separation?) affecting my intercept.
The output is:
<...
- 53
1
vote
0
answers
35
views
estimating effect with marginaleffect package
I want to estimate ATE. first of all I used MatchIt package for full matching for propensity score and then I used logistic regression with all of variable in propensity score model after that I used ...
9
votes
1
answer
316
views
Is G*Power reliable for logistic regression? It does not seem to account for Hauck-Donner
I've just been introduced to G*Power. The only option I could find for analysing logistic regression is described as:
Options: Large sample z-Test, Demidenko (2007) with var corr
That seems to ...
- 929
2
votes
2
answers
323
views
What method should be used if the clusters contains different classes?
Assume that you having $N$ clusters. Each cluster have multiple classes. So we know the class ID for every major clusters, but not the class ID for the data points inside the major clusters.
Each ...
- 415
0
votes
0
answers
15
views
glm.fit: fitted probabilities numerically between 0 and 1 warning [duplicate]
I am currently writing my master's thesis and am doing a logistic regression with the following glm formula in R:
...
Sanne
1
vote
2
answers
116
views
Perfect separation, perhaps? In binary outcome and repeated measure (random effect) with multiple independent variables (using R)
(Using R) - this is my first time posting a stats question online, so please let me know if I'm on the wrong forum or haven't provided enough information and I'll do my best to fix it!
About the data ...
- 11
1
vote
2
answers
508
views
DHARMa residuals plot vs. binned residuals using stan_glm object
I am quite a newbie in R and even more so in Bayesian regression. I have fit a stan_glm binomial model with 1689 observations, 12 variables and two interaction ...
- 13
1
vote
2
answers
748
views
What is the proof that non-linearly separable data can't become linearly separable with the results of PCA?
Give a non-linearly separable dataset $X,$ I want to proof that after performing PCA on it, the resulting dataset is guaranteed to be still non-linearly separable.
I think we could argue that we still ...
- 39
3
votes
2
answers
557
views
fitting a gam with constraints on parameters to deal with separation in parametric terms
I am trying to fit a gam using mgcv which has a mix of smooth and parametric terms. The model is for some count data on fish catches. I am modelling variation in location and time, but also ...
- 41
11
votes
4
answers
3k
views
Do neural networks create a separating hyperplane?
https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ mentions that neural networks learn a representation of the data so as to make the classes linearly separable. What I fail to see is how ...
- 111
0
votes
1
answer
39
views
How to find a mapping to a higher dimension that separates the data, given a data set
We have the following dataset:
$$ \begin{bmatrix}
x_1 & x_2 & y\\
+1 & 0 & +1\\
-1 & 0 & +1\\
0 & +2 & +1\\
0 & +1 & -1
\end{bmatrix} $$
I was asked to find ...
- 35
1
vote
1
answer
117
views
Logistic regression difficulty with multi-level factor and many 0 outcomes
My dataset describes the presence/absence of an animal in a number of different plant species. Below is a table of plant species Vs animal presence/absence, i.e each row show the number of animal ...
- 889
0
votes
0
answers
645
views
Logistic regression - Exp (B) = 0? and sig is 0.999 or 1? [duplicate]
I'm having an issue with binary logistic regression for a project I'm working on.
For some of the variables, I'm receiving a significance value of 0.999 and exp(B) of 0. Is this normal?
6
votes
3
answers
1k
views
Proportion data with number of trials known (and separation?): GLM or beta regression?
I perform a lot of bioassays in which I score mortality not on individuals, but on groups of individuals as a proportion (the denominator, i.e., number of trials, is known):
Sample Data:
...
- 115
3
votes
1
answer
301
views
Unexpected weights in Gradient descent algorithm (linear classification) in python
I am attempting to implement a back propagation algorithm that can efficevley read from a file of features and there targets and predict there outputs correctly, however for sake of testing I am hard ...
- 31
2
votes
0
answers
815
views
Hauck-Donner effect in ordinal generalized regression
I am running a series of ordinal regressions on a rather large data set (n=3640). With some predictors I had issues with violations of parallel lines assumptions, so I decided to run those with a ...
- 33
1
vote
0
answers
45
views
Pooling Profile Penalised LTRs in multiple imputation
I am analysizing data from a clinical trial.
I used multiple imputation to impute the (binary) outcome variable, which is the only variable with missing data.
All of the covariates are categorical and ...
- 11
3
votes
1
answer
435
views
Binomial (1/0) response, Factor explanatory variable: why can't GLM estimate effect when one factor level has all 1s in response?
I have a dataset with a binomial response variable (1/0) and a single explanatory categorical variable with 2 levels.
For one level, the response is all 1s and no 0s (e.g. ...
- 242
2
votes
2
answers
510
views
For the Logistic model, why is the objective function unbounded below if two sets are linearly seperated?
I am reading Approximate linear discrimination via logistic modeling in the Section 8.6.1 of B & V's Convex Optimization book. On Page 428,
$$
\operatorname{minimize} \ -l(a, b) \tag{8.27}
$$
...
- 123
2
votes
0
answers
243
views
Why PCA is not considered in the taxonomy of blind source separation approaches?
Blind source separation (BSS) approaches are divided in the literature into four methods, including independent component analysis (ICA), sparse component analysis (SCA), and non-negative matrix ...
- 216
1
vote
1
answer
60
views
Is the sum of 3 bits a linearly separable task?
In other words can a linear classifier learn to correctly assign a class (label 0 to 3) for an input of 3 bits? Intuitively this cannot work, since the half-adder circuit contains an XOR block, which ...
- 111
8
votes
0
answers
299
views
Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect
In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
- 704
0
votes
0
answers
66
views
On the use of the sampling year as fixed-effect
I have my database in temporal blocks (so I have my occurrence sites into the year of sampling of a few years with maximum 7 years). In order to fix the complete separation problem that I also have, ...
- 155
0
votes
1
answer
88
views
Is there any impact on the results of random forest if we have perfect separation in the database?
I'm having a perfect separation in my database. One of the solutions is to switch randomly some of the values of the dependent variable to the opposite for the variables that cause the problem this ...
- 155
3
votes
1
answer
538
views
Dealing with quasi-complete separation in General Additive Model?
I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
1
vote
0
answers
64
views
How many percentage to randomize and how many iterations in a "what-if analysis"?
I've got complete separated data as such:
...
- 2,331
0
votes
0
answers
135
views
Options when model complexity and separation causes non-convergence in logistic regression
I have created an example data set here
My data represent the presence/absence of a particular animal species (data$outcome) and measurements of trees. I would like ...
- 889
19
votes
3
answers
1k
views
Analysis of Danish mask study data by Nassim Nicholas Taleb (binomial GLM with complete separation)
Recently, Nassim Nicholas Taleb made this post about the recent Danish mask study, a randomized controlled trial which concluded that the proportions of newly diagnosed coronavirus infections was not ...
- 3,329
6
votes
2
answers
839
views
Are these two 2D datasets the same and can they be separated with the XOR NN?
I'm a very beginner in the neural network topic. So I ran into a problem.
I see on the online lectures:
Famous example of a simple non-linearly separable data set, the XOR problem (Minsky 1969) in the ...
0
votes
0
answers
417
views
Is this logistf approach the right way to deal with complete separation in my logistic regression?
I am performing lots of association tests between genotypes and binary diseases. The genotypes can be very rare, and my tests often have extreme case control imbalance (e.g. 500 controls for every ...
- 115
4
votes
1
answer
314
views
Using Chi-squared test when perfect separation in logistic model?
My dataset is similar to this; it describes the presence/absence of a parasite in six species of animal from two locations. Each row in the dataset represents a different individual animal.
I would ...
- 889
1
vote
0
answers
279
views
penalised logistic regression with brglm - warnings
I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
- 63
3
votes
1
answer
566
views
Unexpected behaviour of logit regression with glm in R
I recently was puzzled by the behaviour of R's glm when trying to compute a logistic regression
...
- 33
1
vote
0
answers
36
views
Separation and Correlation
In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
- 229
0
votes
0
answers
646
views
What is the cause of this very high standard error in my logistic regression model?
I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
- 1
3
votes
1
answer
205
views
Probability of linear separation in a dataset with categorical response
I am simulating datasets to which I fit polytomous logistic regression models. The maximum likelihood estimator of this model is undefined when all categories are linearly separated (and it is quite ...
- 2,629
4
votes
1
answer
800
views
What does it means getting p-values equal to 1 and complete separation in logistic regression? [duplicate]
I was really confused if I should ask this here or in Stackoverflow, but I'll give a shot here. I ran a logistic regression using statsmodels library in Python. However, two things went wrong here (...
- 690
0
votes
0
answers
343
views
Separating hardly distinguishable classes for easier classification
I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts.
I have a dataset with a small number of features ...
- 111
2
votes
1
answer
367
views
Logistic Regression assumption
In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
0
votes
1
answer
38
views
Fourier analysis to retrieve components of individual spectra
I have a basic, simple question, I am a physics student, and searching internet gives me a lot of signal processing theory but couldn't find this basic answer, which I plan to implement in my speech ...
- 171
0
votes
0
answers
629
views
Cumulative explained variance between scaled and unscaled data
I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
- 111
2
votes
1
answer
949
views
instability of logistic regression
I am reading Introduction to Statistical Learning and it is said (as in other websites) that Logistic Regression is unstable compared to Linear Discriminant Analysis in well separated cases. ...
- 9,227
1
vote
1
answer
186
views
Doubt on d-separation
In the book: Bayesian Networks With Examples in R, the author shows three examples of d-separation:
He cites:
Then, just a few lines below, the author uses the dsep function, which returns FALSE for ...
- 789