Questions tagged [separation]

Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.

Filter by
Sorted by
Tagged with
0 votes
1 answer
13 views

How to find a mapping to a higher dimension that separates the data, given a data set

We have the following dataset: $$ \begin{bmatrix} x_1 & x_2 & y\\ +1 & 0 & +1\\ -1 & 0 & +1\\ 0 & +2 & +1\\ 0 & +1 & -1 \end{bmatrix} $$ I was asked to find ...
user avatar
1 vote
1 answer
32 views

Logistic regression difficulty with multi-level factor and many 0 outcomes

My dataset describes the presence/absence of an animal in a number of different plant species. Below is a table of plant species Vs animal presence/absence, i.e each row show the number of animal ...
user avatar
0 votes
0 answers
29 views

Logistic regression - Exp (B) = 0? and sig is 0.999 or 1? [duplicate]

I'm having an issue with binary logistic regression for a project I'm working on. For some of the variables, I'm receiving a significance value of 0.999 and exp(B) of 0. Is this normal?
user avatar
5 votes
3 answers
410 views

Proportion data with number of trials known (and separation?): GLM or beta regression?

I perform a lot of bioassays in which I score mortality not on individuals, but on groups of individuals as a proportion (the denominator, i.e., number of trials, is known): Sample Data: ...
user avatar
3 votes
1 answer
52 views

Unexpected weights in Gradient descent algorithm (linear classification) in python

I am attempting to implement a back propagation algorithm that can efficevley read from a file of features and there targets and predict there outputs correctly, however for sake of testing I am hard ...
user avatar
0 votes
0 answers
114 views

Hauck-Donner effect in ordinal generalized regression

I am running a series of ordinal regressions on a rather large data set (n=3640). With some predictors I had issues with violations of parallel lines assumptions, so I decided to run those with a ...
user avatar
  • 1
1 vote
0 answers
20 views

Pooling Profile Penalised LTRs in multiple imputation

I am analysizing data from a clinical trial. I used multiple imputation to impute the (binary) outcome variable, which is the only variable with missing data. All of the covariates are categorical and ...
user avatar
  • 11
1 vote
1 answer
94 views

Binomial (1/0) response, Factor explanatory variable: why can't GLM estimate effect when one factor level has all 1s in response?

I have a dataset with a binomial response variable (1/0) and a single explanatory categorical variable with 2 levels. For one level, the response is all 1s and no 0s (e.g. ...
user avatar
  • 202
0 votes
0 answers
44 views

Is there quasi-seperation here (glmm, logistic regression), and how to take care of it?

I am running a mixed model (logistic regression) using the glmer function of lme4. I have a binary response variable "Germination", one categorical variable "Habitat" with 4 levels ...
user avatar
  • 1
2 votes
2 answers
177 views

For the Logistic model, why is the objective function unbounded below if two sets are linearly seperated?

I am reading Approximate linear discrimination via logistic modeling in the Section 8.6.1 of B & V's Convex Optimization book. On Page 428, $$ \operatorname{minimize} \ -l(a, b) \tag{8.27} $$ ...
user avatar
  • 123
2 votes
0 answers
74 views

Why PCA is not considered in the taxonomy of blind source separation approaches?

Blind source separation (BSS) approaches are divided in the literature into four methods, including independent component analysis (ICA), sparse component analysis (SCA), and non-negative matrix ...
user avatar
0 votes
0 answers
49 views

feature engineering for linear separability and curse of dimensionality

A common method to make otherwise not linear separable datasets separable, is feature engineering - e.g. squaring original feature as show here: Taken from here. Ultimately, a dataset with more ...
user avatar
  • 1,346
0 votes
0 answers
25 views

How to test (overall) significance of categorical variable(with multiple levels) in bias-reduced binomial-response GLM(brglm package), & posthoc-test?

I have a dataset that has quasi-separation. My dependent variable is 'Yes' or 'No', and I have two fixed effects and one random effect. The fixed effect 'Treatment' is a categorical variable and has ...
user avatar
1 vote
0 answers
18 views

Cold Start and First Price Auctions

I have the following contrived scenario... I've participated on various auction platforms where I bid on widgets. Assume that win/loss outcomes on individual platforms are well-separated such that for ...
user avatar
  • 11
1 vote
1 answer
50 views

Is the sum of 3 bits a linearly separable task?

In other words can a linear classifier learn to correctly assign a class (label 0 to 3) for an input of 3 bits? Intuitively this cannot work, since the half-adder circuit contains an XOR block, which ...
user avatar
  • 111
4 votes
0 answers
161 views

Hidden vs Firth vs Shen-Gao logistic regression: dealing with the Hauck-Donner effect

In 1993 a version of penalized logistic regression was introduced by Firth in order to reduce the bias due to outliers and/or (quasi-)perfect prediction in logistic regression: Bias Reduction of ...
user avatar
0 votes
0 answers
24 views

On the use of the sampling year as fixed-effect

I have my database in temporal blocks (so I have my occurrence sites into the year of sampling of a few years with maximum 7 years). In order to fix the complete separation problem that I also have, ...
user avatar
  • 135
0 votes
1 answer
34 views

Is there any impact on the results of random forest if we have perfect separation in the database?

I'm having a perfect separation in my database. One of the solutions is to switch randomly some of the values of the dependent variable to the opposite for the variables that cause the problem this ...
user avatar
  • 135
3 votes
1 answer
261 views

Dealing with quasi-complete separation in General Additive Model?

I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
user avatar
1 vote
0 answers
56 views

How many percentage to randomize and how many iterations in a "what-if analysis"?

I've got complete separated data as such: ...
user avatar
  • 1,751
0 votes
0 answers
59 views

Options when model complexity and separation causes non-convergence in logistic regression

I have created an example data set here My data represent the presence/absence of a particular animal species (data$outcome) and measurements of trees. I would like ...
user avatar
19 votes
3 answers
1k views

Analysis of Danish mask study data by Nassim Nicholas Taleb (binomial GLM with complete separation)

Recently, Nassim Nicholas Taleb made this post about the recent Danish mask study, a randomized controlled trial which concluded that the proportions of newly diagnosed coronavirus infections was not ...
user avatar
4 votes
2 answers
579 views

Are these two 2D datasets the same and can they be separated with the XOR NN?

I'm a very beginner in the neural network topic. So I ran into a problem. I see on the online lectures: Famous example of a simple non-linearly separable data set, the XOR problem (Minsky 1969) in the ...
user avatar
0 votes
0 answers
228 views

Is this logistf approach the right way to deal with complete separation in my logistic regression?

I am performing lots of association tests between genotypes and binary diseases. The genotypes can be very rare, and my tests often have extreme case control imbalance (e.g. 500 controls for every ...
user avatar
  • 133
2 votes
1 answer
182 views

Using Chi-squared test when perfect separation in logistic model?

My dataset is similar to this; it describes the presence/absence of a parasite in six species of animal from two locations. Each row in the dataset represents a different individual animal. I would ...
user avatar
1 vote
0 answers
121 views

penalised logistic regression with brglm - warnings

I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
user avatar
  • 43
3 votes
1 answer
250 views

Unexpected behaviour of logit regression with glm in R

I recently was puzzled by the behaviour of R's glm when trying to compute a logistic regression ...
user avatar
  • 33
1 vote
0 answers
25 views

Separation and Correlation

In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
user avatar
0 votes
0 answers
260 views

What is the cause of this very high standard error in my logistic regression model?

I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
user avatar
  • 1
3 votes
1 answer
140 views

Probability of linear separation in a dataset with categorical response

I am simulating datasets to which I fit polytomous logistic regression models. The maximum likelihood estimator of this model is undefined when all categories are linearly separated (and it is quite ...
user avatar
  • 2,171
3 votes
1 answer
295 views

What does it means getting p-values equal to 1 and complete separation in logistic regression? [duplicate]

I was really confused if I should ask this here or in Stackoverflow, but I'll give a shot here. I ran a logistic regression using statsmodels library in Python. However, two things went wrong here (...
user avatar
  • 610
0 votes
0 answers
93 views

Separating hardly distinguishable classes for easier classification

I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts. I have a dataset with a small number of features ...
user avatar
  • 111
2 votes
1 answer
236 views

Logistic Regression assumption

In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
user avatar
0 votes
1 answer
27 views

Fourier analysis to retrieve components of individual spectra

I have a basic, simple question, I am a physics student, and searching internet gives me a lot of signal processing theory but couldn't find this basic answer, which I plan to implement in my speech ...
user avatar
0 votes
0 answers
293 views

Cumulative explained variance between scaled and unscaled data

I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
user avatar
  • 111
2 votes
1 answer
498 views

instability of logistic regression

I am reading Introduction to Statistical Learning and it is said (as in other websites) that Logistic Regression is unstable compared to Linear Discriminant Analysis in well separated cases. ...
user avatar
  • 7,875
1 vote
1 answer
85 views

Doubt on d-separation

In the book: Bayesian Networks With Examples in R, the author shows three examples of d-separation: He cites: Then, just a few lines below, the author uses the dsep function, which returns FALSE for ...
user avatar
9 votes
0 answers
3k views

Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?

Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
user avatar
  • 570
2 votes
1 answer
82 views

likelihood ratio test for logistic regression with a complete separation

Can likelihood ratio test be used for logistic regression in the case of complete separation? I understand that individual parameters can go to infinity so Wald's test is not recommended, but does ...
user avatar
  • 6,156
1 vote
1 answer
43 views

Clustering Proof of Equation

Greetings! Could anyone enlighten me about the validity of this equation? I'm trying to prove it without success. $K$ is the number of clusters, $C_i$ is the $i$-th cluster, $m_i$ is the number of ...
user avatar
1 vote
1 answer
113 views

Relationship between Cohesion and Separation

Hello! May someone explain to me please how to prove the equation depiicted with the red line? Thank you in advance!
user avatar
2 votes
1 answer
73 views

Doesn't separation in data affects Random Forest?

I'd like to know if separation in data points given by a certain variable will affect Random Forest in terms of "being a correct built model"? For example, logistic regression is highly vulnerable in ...
user avatar
  • 277
5 votes
1 answer
427 views

Why one result is so wide in this logistic multiple regession

I am doing multiple logistic regression with data with 24 predictor variables and 193 rows. All predictor variables have values of 0 or 1 and outcome variables (OUTVAR) also has only 2 possibilities. ...
user avatar
  • 9,133
2 votes
0 answers
63 views

Is A ⊥ B | C where one path active but another inactive?

I'm trying to determine if A ⊥ B | C? I see two paths flowing through elements of C: (1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path) (...
user avatar
  • 216
0 votes
0 answers
285 views

Cutting dendrogram at certain point

I have a question about cutting dendrogram like that It shows some hierarchy in the prison I have to cut it to separate the group. Is it possible to cut not all dendrograph at the same height like ...
user avatar
  • 1
1 vote
0 answers
56 views

What is the *formal* definition of separation in *binary* logistic regression?

I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper https://www.jstor....
user avatar
  • 11
2 votes
1 answer
824 views

Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation

In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R. Although I have read many papers on the Firth ...
user avatar
  • 21
1 vote
1 answer
134 views

Quasi/Complete separation due to huge and infinite values

(R statistics) My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure. I have a example dataset here I managed to run ...
user avatar
4 votes
1 answer
4k views

If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
user avatar
  • 143
0 votes
0 answers
37 views

Linear separation in higher dimension [duplicate]

I am having a problem comprehending with the relation of kernel, weight and linear separation. I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$. And we ...
user avatar