Questions tagged [logistic]
Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression
8,045
questions
381
votes
12
answers
367k
views
Difference between logit and probit models
What is the difference between Logit and Probit model?
I'm more interested here in knowing when to use logistic regression, and when to use Probit.
If there is any literature which defines it using ...
- 6,184
203
votes
10
answers
220k
views
How to deal with perfect separation in logistic regression?
If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message:
...
- 6,981
139
votes
3
answers
273k
views
What is the difference between linear regression and logistic regression?
What is the difference between linear regression and logistic regression?
When would you use each?
- 2,913
124
votes
4
answers
240k
views
Softmax vs Sigmoid function in Logistic classifier?
What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ?
Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
- 1,725
120
votes
3
answers
125k
views
Does an unbalanced sample matter when doing logistic regression?
Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables.
My problem is the following: ...
- 1,303
116
votes
4
answers
45k
views
Why isn't Logistic Regression called Logistic Classification?
Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
- 2,136
114
votes
4
answers
213k
views
What is rank deficiency, and how to deal with it?
Fitting a logistic regression using lme4 ends with
Error in mer_finalize(ans) : Downdated X'X is not positive definite.
A likely cause of this error is ...
- 4,732
102
votes
2
answers
81k
views
Solving for regression parameters in closed-form vs gradient descent
In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method.
I know gradient ...
- 3,605
99
votes
5
answers
151k
views
How to calculate Area Under the Curve (AUC), or the c-statistic, by hand
I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model.
For example, in the validation dataset, I have the true value for the ...
- 3,554
92
votes
5
answers
94k
views
What do the residuals in a logistic regression mean?
In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
- 17.7k
91
votes
4
answers
60k
views
What is the difference between a "link function" and a "canonical link function" for GLM
What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other?
For example, a binary response variable ...
- 1,862
81
votes
1
answer
8k
views
How does a simple logistic regression model achieve a 92% classification accuracy on MNIST?
Even though all the images in the MNIST dataset are centered, with a similar scale, and face up with no rotations, they have a significant handwriting variation that puzzles me how a linear model ...
- 843
79
votes
3
answers
48k
views
Diagnostics for logistic regression?
For linear regression, we can check the diagnostic plots (residuals plots, Normal QQ plots, etc) to check if the assumptions of linear regression are violated.
For logistic regression, I am having ...
- 1,817
75
votes
3
answers
143k
views
Is standardization needed before fitting logistic regression?
My question is do we need to standardize the data set to make sure all variables have the same scale, between [0,1], before fitting logistic regression. The formula is:
$$\frac{x_i-\min(x_i)}{\max(...
- 1,327
71
votes
9
answers
85k
views
Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?
I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
- 13.8k
70
votes
3
answers
78k
views
Why is logistic regression a linear classifier?
Since we are using the logistic function to transform a linear combination of the input into a non-linear output, how can logistic regression be considered a linear classifier?
Linear regression is ...
- 8,091
66
votes
1
answer
161k
views
Wald test for logistic regression
As far as I understand the Wald test in the context of logistic regression is used to determine whether a certain predictor variable $X$ is significant or not. It rejects the null hypothesis of the ...
- 1,501
63
votes
5
answers
124k
views
How to calculate pseudo-$R^2$ from R's logistic regression?
Christopher Manning's writeup on logistic regression in R shows a logistic regression in R as follows:
...
- 3,176
63
votes
4
answers
55k
views
Logistic Regression - Error Term and its Distribution
On whether an error term exists in logistic regression (and its assumed distribution), I have read in various places that:
no error term exists
the error term has a binomial distribution (in ...
- 633
63
votes
4
answers
99k
views
Comparing SVM and logistic regression
Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the ...
- 681
63
votes
1
answer
20k
views
Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? [duplicate]
I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
- 763
63
votes
4
answers
50k
views
Regression for an outcome (ratio or fraction) between 0 and 1
I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$.
I could use linear regression, although it doesn'...
- 3,176
60
votes
5
answers
132k
views
Which loss function is correct for logistic regression?
I read about two versions of the loss function for logistic regression, which of them is correct and why?
From Machine Learning, Zhou Z.H (in Chinese), with $\beta = (w, b)\text{ and }\beta^Tx=w^Tx +...
- 734
59
votes
6
answers
19k
views
Alternatives to logistic regression in R
I would like as many algorithms that perform the same task as logistic regression. That is algorithms/models that can give a prediction to a binary response (Y) with some explanatory variable (X).
...
59
votes
4
answers
50k
views
Why sigmoid function instead of anything else?
Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression?
Why don't we use many of the other derivable functions, with ...
- 855
58
votes
3
answers
118k
views
Logistic Regression: Scikit Learn vs Statsmodels
I am trying to understand why the output from logistic regression of these
two libraries gives different results.
I am using the dataset from UCLA idre tutorial, predicting ...
- 893
57
votes
4
answers
47k
views
Multinomial logistic regression vs one-vs-rest binary logistic regression
Lets say we have a dependent variable $Y$ with few categories and set of independent variables.
What are the advantages of multinomial logistic regression over set of binary logistic regressions (i....
- 3,944
56
votes
2
answers
57k
views
How to simulate artificial data for logistic regression?
I know I'm missing something in my understanding of logistic regression, and would really appreciate any help.
As far as I understand it, the logistic regression assumes that the probability of a '1' ...
- 757
55
votes
7
answers
192k
views
Logistic Regression in R (Odds Ratio)
I'm trying to undertake a logistic regression analysis in R. I have attended courses covering this material using STATA. I am finding it very difficult to replicate ...
- 1,201
55
votes
3
answers
55k
views
Regularization methods for logistic regression
Regularization using methods such as Ridge, Lasso, ElasticNet is quite common for linear regression. I wanted to know the following:
Are these methods applicable for logistic regression? If so, are ...
- 836
54
votes
1
answer
19k
views
Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?
I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
- 740
54
votes
2
answers
126k
views
Interpretation of R's output for binomial regression
I'm quite new on this with binomial data tests, but needed to do one and now I´m not sure how to interpret the outcome. The y-variable, the response variable, is binomial and the explanatory factors ...
- 551
53
votes
1
answer
59k
views
Obtaining predicted values (Y=1 or 0) from a logistic regression model fit
Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
- 1,426
53
votes
4
answers
52k
views
How to do logistic regression subset selection?
I am fitting a binomial family glm in R, and I have a whole troupe of explanatory variables, and I need to find the best (R-squared as a measure is fine). Short of writing a script to loop through ...
- 631
52
votes
3
answers
38k
views
Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?
Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
- 5,908
52
votes
1
answer
117k
views
Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)
I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
- 523
51
votes
3
answers
58k
views
Logistic regression vs. LDA as two-class classifiers
I am trying to wrap my head around the statistical difference between Linear discriminant analysis and Logistic regression. Is my understanding right that, for a two class classification problem, LDA ...
- 2,228
51
votes
1
answer
42k
views
Regression: Transforming Variables
When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in:
Let, $x_1,x_2,x_3$ be age, length of ...
- 6,872
49
votes
2
answers
150k
views
Logistic regression model does not converge
I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
- 1,199
48
votes
13
answers
36k
views
Can machine learning decode the SHA256 hashes?
I have a 64 character SHA256 hash.
I'm hoping to train a model that can predict if the plaintext used to generate the hash begins with a 1 or not.
Regardless if this is "Possible", what algorithm ...
- 521
46
votes
4
answers
163k
views
McFadden's Pseudo-$R^2$ Interpretation
I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this ...
- 3,554
46
votes
2
answers
34k
views
Poisson regression to estimate relative risk for binary outcomes
Brief Summary
Why is it more common for logistic regression (with odds ratios) to be used in cohort studies with binary outcomes, as opposed to Poisson regression (with relative risks)?
Background
...
- 2,317
45
votes
3
answers
63k
views
Calculating confidence intervals for a logistic regression
I'm using a binomial logistic regression to identify if exposure to has_x or has_y impacts the likelihood that a user will click ...
- 1,484
45
votes
1
answer
13k
views
Does down-sampling change logistic regression coefficients?
If I have a dataset with a very rare positive class, and I down-sample the negative class, then perform a logistic regression, do I need to adjust the regression coefficients to reflect the fact that ...
- 22.7k
44
votes
5
answers
57k
views
What is the significance of logistic regression coefficients?
I am currently reading a paper concerning voting location and voting preference in the 2000 and 2004 election. In it, there is a chart which displays logistic regression coefficients. From courses ...
- 543
43
votes
2
answers
24k
views
Simulation of logistic regression power analysis - designed experiments
This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
- 7,940
43
votes
2
answers
21k
views
Why is logistic regression a linear model?
I want to know why logistic regression is called a linear model. It uses a sigmoid function, which is not linear. So why is logistic regression a linear model?
- 6,337
42
votes
3
answers
27k
views
Logistic Regression: Bernoulli vs. Binomial Response Variables
I want to perform logistic regression with the following binomial response and with $X_1$ and $X_2$ as my predictors.
I can present the same data as Bernoulli responses in the following format.
The ...
- 583
41
votes
2
answers
68k
views
Interpretation of plot (glm.model)
Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
- 411