Questions tagged [separation]
Separation occurs when some classes of a categorical outcome can be perfectly distinguished by a linear combination of other variables.
151
questions
3
votes
1answer
58 views
Dealing with quasi-complete separation in General Additive Model?
I am modelling influence of fire on occurrence of certain bird species (count response variable) in Before/after control/impact experiment design. I've got intact and burned sites and count data both ...
1
vote
0answers
49 views
How many percentage to randomize and how many iterations in a “what-if analysis”?
I've got complete separated data as such:
...
0
votes
0answers
23 views
Options when model complexity and separation causes non-convergence in logistic regression
I have created an example data set here
My data represent the presence/absence of a particular animal species (data$outcome) and measurements of trees. I would like ...
18
votes
3answers
642 views
Analysis of Danish mask study data by Nassim Nicholas Taleb (binomial GLM with complete separation)
Recently, Nassim Nicholas Taleb made this post about the recent Danish mask study, a randomized controlled trial which concluded that the proportions of newly diagnosed coronavirus infections was not ...
4
votes
2answers
270 views
Are these two 2D datasets the same and can they be separated with the XOR NN?
I'm a very beginner in the neural network topic. So I ran into a problem.
I see on the online lectures:
Famous example of a simple non-linearly separable data set, the XOR problem (Minsky 1969) in the ...
2
votes
1answer
84 views
Using Chi-squared test when perfect separation in logistic model?
My dataset is similar to this; it describes the presence/absence of a parasite in six species of animal from two locations. Each row in the dataset represents a different individual animal.
I would ...
1
vote
0answers
40 views
penalised logistic regression with brglm - warnings
I've built a logistic model to predict a binary response. I've got four categorical predictors. One of them (posicion) has 6 levels, 3 of which occur not too frequently and are ALWAYS (by definition) ...
0
votes
0answers
48 views
How to use the kernel trick on a XOR-like dataset
Let's say that I have the following data:
I want to find a transformation of this dataset that will make it linearly separable.
My thought was to bring the data around the origin and then multiply $...
0
votes
0answers
8 views
Identify two points from a set as similar
I'm trying to build a model to asses different physiological parameters (Dry weight, yield, Nitrogen concentration, one model for each) from a field based on reflectance from different wavelengths i.e....
1
vote
0answers
21 views
Separation and Correlation
In a logistic regression setting, can there be a relation between the fact that covariates are highly correlated and the problem of perfect(quasi-perfect) separation?
0
votes
0answers
118 views
What is the cause of this very high standard error in my logistic regression model?
I am running a logistic regression model where the outcome variable is Neurologic Complications, and there are various factors who's impact I am examining. One of the factors (HTN_new1), a categorical ...
3
votes
1answer
84 views
Probability of linear separation in a dataset with categorical response
I am simulating datasets to which I fit polytomous logistic regression models. The maximum likelihood estimator of this model is undefined when all categories are linearly separated (and it is quite ...
0
votes
0answers
39 views
1D representation for 2D toy data (about linear separability)
suppose there is a dataset with 2 features x1, and x2. the points (-1;-1); (1; 1); (-3;-3); (4; 4) belong to class 1 and (-1; 1); (1;-1); (-5; 2); (4;-8) belongs to class 2.
I am confused in terms of ...
0
votes
0answers
27 views
Separating hardly distinguishable classes for easier classification
I apologize for this seemingly basic question, but I was just thrust into this data science role and would like to get some advice from some experts.
I have a dataset with a small number of features ...
1
vote
1answer
85 views
Logistic Regression assumption
In Logistic Regression, the assumption is that the data must be linearly separable, but if the data is not linearly separable then we can't apply Logistic Regression?
0
votes
1answer
20 views
Fourier analysis to retrieve components of individual spectra
I have a basic, simple question, I am a physics student, and searching internet gives me a lot of signal processing theory but couldn't find this basic answer, which I plan to implement in my speech ...
0
votes
0answers
87 views
Cumulative explained variance between scaled and unscaled data
I was writing a small piece of code to portray the difference between scaled and unscaled data when doing PCA and (as expected) I found that the separability of classes was better by doing the ...
1
vote
1answer
122 views
instability of logistic regression
I am reading Introduction to Statistical Learning and it is said (as in other websites) that Logistic Regression is unstable compared to Linear Discriminant Analysis in well separated cases. ...
0
votes
0answers
21 views
Preprocessing on unsupervised learning
I am working on a high dimensional problem that evaluates code readability according to specific metrics. The problem is that there is no 'ground truth' so I need to implement clustering (instead of ...
0
votes
1answer
52 views
Doubt on d-separation
In the book: Bayesian Networks With Examples in R, the author shows three examples of d-separation:
He cites:
Then, just a few lines below, the author uses the dsep function, which returns FALSE for ...
6
votes
0answers
1k views
Which theorem in Cover's 1965 paper is actually referred to as Cover's Theorem?
Cover's Theorem is stated on Wikipedia (and similarly elsewhere) as
A complex pattern-classification problem, cast in a high-dimensional space nonlinearly, is more likely to be linearly separable ...
2
votes
0answers
34 views
likelihood ratio test for logistic regression with a complete separation
Can likelihood ratio test be used for logistic regression in the case of complete separation? I understand that individual parameters can go to infinity so Wald's test is not recommended, but does ...
1
vote
1answer
38 views
Clustering Proof of Equation
Greetings! Could anyone enlighten me about the validity of this equation? I'm trying to prove it without success. $K$ is the number of clusters, $C_i$ is the $i$-th cluster, $m_i$ is the number of ...
1
vote
1answer
26 views
Relationship between Cohesion and Separation
Hello! May someone explain to me please how to prove the equation depiicted with the red line? Thank you in advance!
1
vote
1answer
40 views
Doesn't separation in data affects Random Forest?
I'd like to know if separation in data points given by a certain variable will affect Random Forest in terms of "being a correct built model"? For example, logistic regression is highly vulnerable in ...
5
votes
1answer
408 views
Why one result is so wide in this logistic multiple regession
I am doing multiple logistic regression with data with 24 predictor variables and 193 rows.
All predictor variables have values of 0 or 1 and outcome variables (OUTVAR) also has only 2 possibilities. ...
2
votes
0answers
52 views
Is A ⊥ B | C where one path active but another inactive?
I'm trying to determine if A ⊥ B | C?
I see two paths flowing through elements of C:
(1) B <- C - > A (all variables unobserved, active triple; independence cannot be guaranteed for this path)
(...
0
votes
0answers
60 views
Cutting dendrogram at certain point
I have a question about cutting dendrogram like that
It shows some hierarchy in the prison
I have to cut it to separate the group.
Is it possible to cut not all dendrograph at the same height like ...
1
vote
0answers
55 views
What is the *formal* definition of separation in *binary* logistic regression?
I am trying to understand complete and quasi complete separation in the context of logistic binary regression. However, I have not found a clear source. I know the seminal paper
https://www.jstor....
2
votes
1answer
189 views
Seeking to understand using the Firth correction in Generalized Estimating Equations to deal with quasi-complete separation
In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R.
Although I have read many papers on the Firth ...
1
vote
1answer
71 views
Quasi/Complete separation due to huge and infinite values
(R statistics)
My question is regarding this warning. My data contains patients and healthy subjects. Exponential decay is my outcome measure.
I have a example dataset here
I managed to run ...
1
vote
1answer
1k views
If logistic regression is a linear classifier why does it fail on linearly separable data?
Logistic regression is a linear model, decision boundary generated is linear.
If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
0
votes
0answers
36 views
Linear separation in higher dimension [duplicate]
I am having a problem comprehending with the relation of kernel, weight and linear separation.
I have a case where I am given a kernel $k_1$. that has a corresponding mapping $\phi_1$.
And we ...
1
vote
0answers
45 views
anomaly detection : check the separability of normal and abnormal data
I'd like to develop an anomaly detection. I have historical data from sensors in the form of time series. The time series can be divided into data of a normal state and data of an abnormal state i.e. ...
2
votes
1answer
225 views
Handling quasi-perfect separation in a zero-inflated negative binomial regression in R
I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
0
votes
1answer
40 views
Dealing with Complete Separation in Logistic Regression when Reporting [duplicate]
I have a question regarding how one would deal with complete separation in logistic regression when reporting the outcome for statistical analysis. For a study, we have group participants into 4 ...
5
votes
1answer
17k views
glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric
I've been receiving the warning message in the title and have reviewed posts such as e.g. this one.
I would like to understand how this feature has perfect separation with the target variable, since ...
1
vote
0answers
35 views
Does separation of data matter in Bayesian power calculations with logistic regression models?
I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
0
votes
1answer
208 views
A priori contrast for binomial GLM
after much reading I decided to write because I cannot find a solution to my question.
I already did a priori contrasts before for a continuous variable with normal distribution. Now I have another ...
1
vote
1answer
69 views
Is there an upper bound on number of logistic regression responses that yield infinite estimates
Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
0
votes
0answers
99 views
Logistic regression with separation and nested design in R
Data:
An item can be in the state 0 or 1 (binary). Each year, it starts in the state 0 and then changes to state 1.
I have data of 4 seasons (2010 -2013). Each year I sampled the (same) individuals ...
5
votes
2answers
307 views
GLMM for count data using square root link in lme4
I have data from a field survey. The objective of the study is to relate number of seedling (respond variable, count data), landform (exploratory variable, categorical variable with 3 levels) and ...
0
votes
1answer
172 views
Kullback-Leibler Divergence as the class separation measure
I am trying to use KL divergence as the separation measure between the classes. I have the positive, negative samples for 2 distributions and want to adjust the algorithm parameters to get the best ...
1
vote
0answers
14 views
any problems with Firths Logit model (to deal with separation)
I am performing a Logit Model on binary variable SALE, when running on stata some of my explanatory variables were dropped as they were "perfect predictors". after some reading i believe the problem ...
1
vote
1answer
326 views
Zero Centering Data
Does zero centering your data make it more linearly separable or does that not affect the separability? I read that by zero-centering the data, it allows algorithms to train faster because small ...
1
vote
1answer
508 views
Can I run a logistic regression with a small sample size and perfect convergence?
I'm hoping someone will be able to help me with my question. I have a dataset with 71 records and 4 variables. Three of the four variables are independent, and one is dependent. All four variables are ...
2
votes
1answer
99 views
R/Stata Can I include a categorical variable in a Logistic regression when one of its values perfectly predicts one of the outcomes?
Suppose I have the following data
df = data.table('y'= c(1,1,1,1,1,1,1,0,0,0,0), 'x' = c(1,1,1,1,1,1,1,1,1,0,0))
where x = 0 perfectly predicts y = 0. I ...
4
votes
1answer
1k views
How to tell which variable is perfectly separated in R [duplicate]
I am running a logistic regression model in R using glm. I received a warning that complete separation occurred. How do I determine which variable is causing this? R doesn't tell you what variable is ...
1
vote
1answer
464 views
Why do we sum the cost function in a logistic regression?
I get the intuition behind the loss function for linear regression, which happens to be the MSE function. We have a set of points and we try to fit a line between them so that the line is at a minimum ...
1
vote
1answer
1k views
Abnormally high standard error in binary logistic regression
I did a binary logistic regression test on SPSS with a sample size of 24. The study is about the correlation of teenage pregnancy and depressive symptoms. Of the 24, 11 are pregnant teenagers. Of the ...
