Skip to main content

Questions tagged [logistic]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
395 votes
12 answers

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,386
211 votes
9 answers

How to deal with perfect separation in logistic regression?

If you have a variable which perfectly separates zeroes and ones in target variable, R will yield the following "perfect or quasi perfect separation" warning message: ...
user333's user avatar
  • 7,271
140 votes
3 answers

What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
B Seven's user avatar
  • 2,933
132 votes
4 answers

Softmax vs Sigmoid function in Logistic classifier?

What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ? Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
mach's user avatar
  • 1,815
125 votes
3 answers

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
Michiel's user avatar
  • 1,353
119 votes
4 answers

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
Ismael Ghalimi's user avatar
116 votes
4 answers

What is rank deficiency, and how to deal with it?

Fitting a logistic regression using lme4 ends with Error in mer_finalize(ans) : Downdated X'X is not positive definite. A likely cause of this error is ...
Jack Tanner's user avatar
  • 4,872
106 votes
3 answers

Solving for regression parameters in closed-form vs gradient descent

In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method. I know gradient ...
Jeff's user avatar
  • 3,997
103 votes
5 answers

How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model. For example, in the validation dataset, I have the true value for the ...
Matt Reichenbach's user avatar
98 votes
4 answers

What is the difference between a "link function" and a "canonical link function" for GLM

What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other? For example, a binary response variable ...
steadyfish's user avatar
  • 1,962
94 votes
5 answers

What do the residuals in a logistic regression mean?

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
russellpierce's user avatar
85 votes
2 answers

How does a simple logistic regression model achieve a 92% classification accuracy on MNIST?

Even though all the images in the MNIST dataset are centered, with a similar scale, and face up with no rotations, they have a significant handwriting variation that puzzles me how a linear model ...
Nitish Agarwal's user avatar
80 votes
3 answers

Diagnostics for logistic regression?

For linear regression, we can check the diagnostic plots (residuals plots, Normal QQ plots, etc) to check if the assumptions of linear regression are violated. For logistic regression, I am having ...
ialm's user avatar
  • 1,827
78 votes
4 answers

Is standardization needed before fitting logistic regression?

My question is do we need to standardize the data set to make sure all variables have the same scale, between [0,1], before fitting logistic regression. The formula is: $$\frac{x_i-\min(x_i)}{\max(...
user1946504's user avatar
  • 1,357
73 votes
3 answers

Logistic Regression - Error Term and its Distribution

On whether an error term exists in logistic regression (and its assumed distribution), I have read in various places that: no error term exists the error term has a binomial distribution (in ...
user61124's user avatar
  • 733
72 votes
9 answers

Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?

I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
Henrik's user avatar
  • 14.3k
72 votes
3 answers

Why is logistic regression a linear classifier?

Since we are using the logistic function to transform a linear combination of the input into a non-linear output, how can logistic regression be considered a linear classifier? Linear regression is ...
Jack Twain's user avatar
  • 8,421
71 votes
1 answer

Wald test for logistic regression

As far as I understand the Wald test in the context of logistic regression is used to determine whether a certain predictor variable $X$ is significant or not. It rejects the null hypothesis of the ...
user695652's user avatar
  • 1,601
68 votes
4 answers

Comparing SVM and logistic regression

Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the ...
user41799's user avatar
  • 731
67 votes
4 answers

Regression for an outcome (ratio or fraction) between 0 and 1

I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$. I could use linear regression, although it doesn'...
dfrankow's user avatar
  • 3,386
65 votes
5 answers

How to calculate pseudo-$R^2$ from R's logistic regression?

Christopher Manning's writeup on logistic regression in R shows a logistic regression in R as follows: ...
dfrankow's user avatar
  • 3,386
64 votes
1 answer

Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? [duplicate]

I'm trying to predict a binary outcome using 50 continuous explanatory variables (the range of most of the variables is $-\infty$ to $\infty$). My data set has almost 24,000 rows. When I run ...
Dcook's user avatar
  • 773
63 votes
4 answers

Why sigmoid function instead of anything else?

Why is the de-facto standard sigmoid function, $\frac{1}{1+e^{-x}}$, so popular in (non-deep) neural-networks and logistic regression? Why don't we use many of the other derivable functions, with ...
Mark Horvath's user avatar
61 votes
6 answers

Which loss function is correct for logistic regression?

I read about two versions of the loss function for logistic regression, which of them is correct and why? From Machine Learning, Zhou Z.H (in Chinese), with $\beta = (w, b)\text{ and }\beta^Tx=w^Tx +...
xtt's user avatar
  • 744
60 votes
3 answers

Why is there a difference between manually calculating a logistic regression 95% confidence interval, and using the confint() function in R?

Dear everyone - I've noticed something strange that I can't explain, can you? In summary: the manual approach to calculating a confidence interval in a logistic regression model, and the R function <...
Andrew's user avatar
  • 6,278
59 votes
7 answers

Logistic Regression in R (Odds Ratio)

I'm trying to undertake a logistic regression analysis in R. I have attended courses covering this material using STATA. I am finding it very difficult to replicate ...
SabreWolfy's user avatar
  • 1,261
59 votes
6 answers

Alternatives to logistic regression in R

I would like as many algorithms that perform the same task as logistic regression. That is algorithms/models that can give a prediction to a binary response (Y) with some explanatory variable (X). ...
59 votes
3 answers

Logistic Regression: Scikit Learn vs Statsmodels

I am trying to understand why the output from logistic regression of these two libraries gives different results. I am using the dataset from UCLA idre tutorial, predicting ...
hurrikale's user avatar
  • 893
59 votes
4 answers

Multinomial logistic regression vs one-vs-rest binary logistic regression

Lets say we have a dependent variable $Y$ with few categories and set of independent variables. What are the advantages of multinomial logistic regression over set of binary logistic regressions (i....
Tomek Tarczynski's user avatar
58 votes
1 answer

Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
SniperBro2000's user avatar
58 votes
2 answers

Interpretation of R's output for binomial regression

I'm quite new on this with binomial data tests, but needed to do one and now I´m not sure how to interpret the outcome. The y-variable, the response variable, is binomial and the explanatory factors ...
user40116's user avatar
  • 701
57 votes
2 answers

How to simulate artificial data for logistic regression?

I know I'm missing something in my understanding of logistic regression, and would really appreciate any help. As far as I understand it, the logistic regression assumes that the probability of a '1' ...
zorbar's user avatar
  • 767
56 votes
1 answer

Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)

I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
StreetHawk's user avatar
56 votes
3 answers

Regularization methods for logistic regression

Regularization using methods such as Ridge, Lasso, ElasticNet is quite common for linear regression. I wanted to know the following: Are these methods applicable for logistic regression? If so, are ...
Tapan Khopkar's user avatar
54 votes
3 answers

Logistic regression vs. LDA as two-class classifiers

I am trying to wrap my head around the statistical difference between Linear discriminant analysis and Logistic regression. Is my understanding right that, for a two class classification problem, LDA ...
user1885116's user avatar
  • 2,348
54 votes
1 answer

Obtaining predicted values (Y=1 or 0) from a logistic regression model fit

Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
tetragrammaton's user avatar
54 votes
4 answers

How to do logistic regression subset selection?

I am fitting a binomial family glm in R, and I have a whole troupe of explanatory variables, and I need to find the best (R-squared as a measure is fine). Short of writing a script to loop through ...
Leendert's user avatar
  • 641
53 votes
1 answer

Regression: Transforming Variables

When transforming variables, do you have to use all of the same transformation? For example, can I pick and choose differently transformed variables, as in: Let, $x_1,x_2,x_3$ be age, length of ...
Brandon Bertelsen's user avatar
50 votes
13 answers

Can machine learning decode the SHA256 hashes?

I have a 64 character SHA256 hash. I'm hoping to train a model that can predict if the plaintext used to generate the hash begins with a 1 or not. Regardless if this is "Possible", what algorithm ...
John's user avatar
  • 541
50 votes
2 answers

Logistic regression model does not converge

I've got some data about airline flights (in a data frame called flights) and I would like to see if the flight time has any effect on the probability of a ...
Daniel Standage's user avatar
50 votes
2 answers

Poisson regression to estimate relative risk for binary outcomes

Brief Summary Why is it more common for logistic regression (with odds ratios) to be used in cohort studies with binary outcomes, as opposed to Poisson regression (with relative risks)? Background ...
jthetzel's user avatar
  • 2,447
49 votes
5 answers

McFadden's Pseudo-$R^2$ Interpretation

I have a binary logistic regression model with a McFadden's pseudo R-squared of 0.192 with a dependent variable called payment (1 = payment and 0 = no payment). What is the interpretation of this ...
Matt Reichenbach's user avatar
48 votes
1 answer

Does down-sampling change logistic regression coefficients?

If I have a dataset with a very rare positive class, and I down-sample the negative class, then perform a logistic regression, do I need to adjust the regression coefficients to reflect the fact that ...
Zach's user avatar
  • 24.1k
47 votes
3 answers

What does the logit value actually mean?

I have a logit model that comes up with a number between 0 and 1 for many cases, but how can we interprete this? Lets take a case with a logit of 0.20 Can we assert that there is 20% probability ...
Dez's user avatar
  • 471
47 votes
3 answers

Calculating confidence intervals for a logistic regression

I'm using a binomial logistic regression to identify if exposure to has_x or has_y impacts the likelihood that a user will click ...
celenius's user avatar
  • 1,542
47 votes
2 answers

Why is logistic regression a linear model?

I want to know why logistic regression is called a linear model. It uses a sigmoid function, which is not linear. So why is logistic regression a linear model?
user34790's user avatar
  • 6,837
47 votes
3 answers

Logistic Regression: Bernoulli vs. Binomial Response Variables

I want to perform logistic regression with the following binomial response and with $X_1$ and $X_2$ as my predictors. I can present the same data as Bernoulli responses in the following format. The ...
A Scientist's user avatar
45 votes
2 answers

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
B_Miner's user avatar
  • 8,810
45 votes
6 answers

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
GrowinMan's user avatar
  • 961
44 votes
2 answers

Interpretation of plot (glm.model)

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
Summer's user avatar
  • 441

2 3 4 5