Questions tagged [logistic]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
326
votes
10answers
303k views

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
178
votes
9answers
175k views

How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
126
votes
3answers
265k views

What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
91
votes
4answers
164k views

What is rank deficiency, and how to deal with it?

Fitting a logistic regression using lme4 ends with Error in mer_finalize(ans) : Downdated X'X is not positive definite. A likely cause of this error is ...
91
votes
5answers
123k views

How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model. For example, in the validation dataset, I have the true value for the ...
88
votes
3answers
84k views

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
83
votes
3answers
24k views

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
80
votes
2answers
49k views

Solving for regression parameters in closed-form vs gradient descent

In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method. I know gradient ...
77
votes
4answers
115k views

Softmax vs Sigmoid function in Logistic classifier?

What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ? Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
75
votes
1answer
4k views

How does a simple logistic regression model achieve a 92% classification accuracy on MNIST?

Even though all the images in the MNIST dataset are centered, with a similar scale, and face up with no rotations, they have a significant handwriting variation that puzzles me how a linear model ...
74
votes
3answers
34k views

Diagnostics for logistic regression?

For linear regression, we can check the diagnostic plots (residuals plots, Normal QQ plots, etc) to check if the assumptions of linear regression are violated. For logistic regression, I am having ...
68
votes
4answers
39k views

What is the difference between a “link function” and a “canonical link function” for GLM

What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other? For example, a binary response variable ...
66
votes
4answers
67k views

What do the residuals in a logistic regression mean?

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
60
votes
7answers
72k views

Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?

I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
57
votes
6answers
18k views

Alternatives to logistic regression in R

I would like as many algorithms that perform the same task as logistic regression. That is algorithms/models that can give a prediction to a binary response (Y) with some explanatory variable (X). ...
57
votes
1answer
15k views

Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what?

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
56
votes
1answer
126k views

Wald test for logistic regression

As far as I understand the Wald test in the context of logistic regression is used to determine whether a certain predictor variable $X$ is significant or not. It rejects the null hypothesis of the ...
55
votes
3answers
55k views

Why is logistic regression a linear classifier?

Since we are using the logistic function to transform a linear combination of the input into a non-linear output, how can logistic regression be considered a linear classifier? Linear regression is ...
52
votes
1answer
56k views

Obtaining predicted values (Y=1 or 0) from a logistic regression model fit

Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
51
votes
5answers
85k views

How to calculate pseudo-$R^2$ from R's logistic regression?

Christopher Manning's writeup on logistic regression in R shows a logistic regression in R as follows: ...
49
votes
4answers
43k views

How to do logistic regression subset selection?

I am fitting a binomial family glm in R, and I have a whole troupe of explanatory variables, and I need to find the best (R-squared as a measure is fine). Short of writing a script to loop through ...
48
votes
2answers
44k views

How to simulate artificial data for logistic regression?

I know I'm missing something in my understanding of logistic regression, and would really appreciate any help. As far as I understand it, the logistic regression assumes that the probability of a '1' ...
47
votes
3answers
75k views

Is standardization needed before fitting logistic regression?

My question is do we need to standardize the data set to make sure all variables have the same scale, between [0,1], before fitting logistic regression. The formula is: $$\frac{x_i-\min(x_i)}{\max(...
47
votes
4answers
65k views

Comparing SVM and logistic regression

Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the ...
47
votes
4answers
27k views

Regression for an outcome (ratio or fraction) between 0 and 1

I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$. I could use linear regression, although it doesn'...
46
votes
1answer
35k views

Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
46
votes
2answers
116k views

Logistic regression model does not converge

I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
45
votes
3answers
42k views

Regularization methods for logistic regression

Regularization using methods such as Ridge, Lasso, ElasticNet is quite common for linear regression. I wanted to know the following: Are these methods applicable for logistic regression? If so, are ...
44
votes
13answers
23k views

Can machine learning decode the SHA256 hashes?

I have a 64 character SHA256 hash. I'm hoping to train a model that can predict if the plaintext used to generate the hash begins with a 1 or not. Regardless if this is "Possible", what algorithm ...
44
votes
5answers
136k views

Logistic Regression in R (Odds Ratio)

I'm trying to undertake a logistic regression analysis in R. I have attended courses covering this material using STATA. I am finding it very difficult to replicate ...
44
votes
1answer
11k views

Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
44
votes
3answers
33k views

Multinomial logistic regression vs one-vs-rest binary logistic regression

Lets say we have a dependent variable $Y$ with few categories and set of independent variables. What are the advantages of multinomial logistic regression over set of binary logistic regressions (i....
44
votes
2answers
25k views

Poisson regression to estimate relative risk for binary outcomes

Brief Summary Why is it more common for logistic regression (with odds ratios) to be used in cohort studies with binary outcomes, as opposed to Poisson regression (with relative risks)? Background ...
43
votes
5answers
53k views

What is the significance of logistic regression coefficients?

I am currently reading a paper concerning voting location and voting preference in the 2000 and 2004 election. In it, there is a chart which displays logistic regression coefficients. From courses ...
43
votes
4answers
34k views

Why sigmoid function instead of anything else?

Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression? Why don't we use many of the other derivable functions, with ...
42
votes
2answers
95k views

Interpretation of R's output for binomial regression

I'm quite new on this with binomial data tests, but needed to do one and now I´m not sure how to interpret the outcome. The y-variable, the response variable, is binomial and the explanatory factors ...
41
votes
1answer
80k views

Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)

I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
41
votes
3answers
75k views

Logistic Regression: Scikit Learn vs Statsmodels

I am trying to understand why the output from logistic regression of these two libraries gives different results. I am using the dataset from UCLA idre tutorial, predicting ...
40
votes
4answers
88k views

Which loss function is correct for logistic regression?

I read about two versions of the loss function for logistic regression, which of them is correct and why? From Machine Learning, Zhou Z.H (in Chinese), with $\beta = (w, b)\text{ and }\beta^Tx=w^Tx +...
40
votes
4answers
33k views

Logistic Regression - Error Term and its Distribution

On whether an error term exists in logistic regression (and its assumed distribution), I have read in various places that: no error term exists the error term has a binomial distribution (in ...
40
votes
3answers
40k views

Logistic regression vs. LDA as two-class classifiers

I am trying to wrap my head around the statistical difference between Linear discriminant analysis and Logistic regression. Is my understanding right that, for a two class classification problem, LDA ...
39
votes
3answers
24k views

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
39
votes
2answers
18k views

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
36
votes
3answers
18k views

Logistic Regression: Bernoulli vs. Binomial Response Variables

I want to perform logistic regression with the following binomial response and with $X_1$ and $X_2$ as my predictors. I can present the same data as Bernoulli responses in the following format. ...
35
votes
4answers
11k views

Why does logistic regression become unstable when classes are well-separated?

Why is it that logistic regression becomes unstable when classes are well-separated? What does well-separated classes mean? I would really appreciate if someone can explain with an example.
35
votes
1answer
9k views

Does down-sampling change logistic regression coefficients?

If I have a dataset with a very rare positive class, and I down-sample the negative class, then perform a logistic regression, do I need to adjust the regression coefficients to reflect the fact that ...
35
votes
4answers
32k views

ANOVA on binomial data

I am analyzing an experimental data set. The data consists of a paired vector of treatment type and a binomial outcome: ...
33
votes
2answers
49k views

Interpretation of plot (glm.model)

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
33
votes
6answers
53k views

What is the difference between logistic regression and neural networks?

How do we explain the difference between logistic regression and neural network to an audience that have no background in statistics?
33
votes
2answers
4k views

Degrees of freedom of $\chi^2$ in Hosmer-Lemeshow test

The test statistic for the Hosmer-Lemeshow test (HLT) for goodness of fit (GOF) of a logistic regression model is defined as follows: The sample is then split into $d=10$ deciles, $D_1, D_2, \dots ...

1
2 3 4 5
129