Questions tagged [logistic]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
35
votes
7answers
21k views

Are there algorithms for computing “running” linear or logistic regression parameters?

A paper "Accurately computing running variance" at http://www.johndcook.com/standard_deviation.html shows how to compute running mean, variance and standard deviations. Are there algorithms where the ...
35
votes
2answers
21k views

When is logistic regression solved in closed form?

Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form? One ...
34
votes
6answers
33k views

What's the difference between logistic regression and perceptron?

I'm going through Andrew Ng's lecture notes on Machine Learning. The notes introduce us to logistic regression and then to perceptron. While describing Perceptron, the notes say that we just change ...
33
votes
6answers
55k views

What is the difference between logistic regression and neural networks?

How do we explain the difference between logistic regression and neural network to an audience that have no background in statistics?
33
votes
2answers
28k views

Interpretation of simple predictions to odds ratios in logistic regression

I'm somewhat new to using logistic regression, and a bit confused by a discrepancy between my interpretations of the following values which I thought would be the same: exponentiated beta values ...
33
votes
3answers
21k views

Kernel logistic regression vs SVM

As is known to all, SVM can use kernel method to project data points in higher spaces so that points can be separated by a linear space. But we can also use logistic regression to choose this ...
33
votes
2answers
4k views

Degrees of freedom of $\chi^2$ in Hosmer-Lemeshow test

The test statistic for the Hosmer-Lemeshow test (HLT) for goodness of fit (GOF) of a logistic regression model is defined as follows: The sample is then split into $d=10$ deciles, $D_1, D_2, \dots ...
32
votes
3answers
37k views

What does the logit value actually mean?

I have a logit model that comes up with a number between 0 and 1 for many cases, but how can we interprete this? Lets take a case with a logit of 0.20 Can we assert that there is 20% probability ...
32
votes
5answers
40k views

Overfitting a logistic regression model

Is it possible to overfit a logistic regression model? I saw a video saying that if my area under the ROC curve is higher than 95%, then its very likely to be over fitted, but is it possible to ...
32
votes
3answers
13k views

Why is logistic regression a linear model?

I want to know why logistic regression is called a linear model. It uses a sigmoid function, which is not linear. So why is logistic regression a linear model?
32
votes
1answer
26k views

How are the standard errors computed for the fitted values from a logistic regression?

When you predict a fitted value from a logistic regression model, how are standard errors computed? I mean for the fitted values, not for the coefficients (which involves Fishers information matrix). ...
32
votes
5answers
23k views

Strategy to deal with rare events logistic regression

I would like to study rare events in a finite population. Since I am unsure about which strategy is best suited, I would appreciate tips and references related to this matter, although I am well-aware ...
30
votes
6answers
98k views

Sample size for logistic regression?

I want to make a logistic model from my survey data. It is a small survey of four residential colonies in which only 154 respondents were interviewed. My dependent variable is "satisfactory transition ...
30
votes
2answers
159k views

Likelihood ratio test in R

Suppose I am going to do a univariate logistic regression on several independent variables, like this: ...
30
votes
3answers
46k views

What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
29
votes
3answers
23k views

How to do logistic regression in R when outcome is fractional (a ratio of two counts)?

I'm reviewing a paper which has the following biological experiment. A device is used to expose cells to varying amounts of fluid shear stress. As greater shear stress is applied to the cells, more of ...
28
votes
1answer
25k views

What is the difference between generalized estimating equations and GLMM?

I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
28
votes
3answers
40k views

How to understand output from R's polr function (ordered logistic regression)?

I am new to R, ordered logistic regression, and polr. The "Examples" section at the bottom of the help page for polr (that fits a logistic or probit regression ...
28
votes
3answers
8k views

Intuition behind logistic regression

Recently I began studying machine learning, however I failed to grasp the intuition behind logistic regression. The following are the facts about logistic regression that I understand. As the basis ...
27
votes
9answers
60k views

Measuring accuracy of a logistic regression-based model

I have a trained logistic regression model that I am applying to a testing data set. The dependent variable is binary (boolean). For each sample in the testing data set, I apply the logistic ...
27
votes
4answers
23k views

What is the relationship between regression and linear discriminant analysis (LDA)?

Is there a relationship between regression and linear discriminant analysis (LDA)? What are their similarities and differences? Does it make any difference if there are two classes or more than two ...
27
votes
2answers
23k views

How to use ordinal logistic regression with random effects?

In my study I will be measuring workload with several metrics. With heart-rate variability (HRV), electrodermal activity (EDA) and with a subjective scale (IWS). After normalization the IWS has three ...
27
votes
3answers
52k views

Interpreting interaction terms in logit regression with categorical variables

I have data from a survey experiment in which respondents were randomly assigned to one of four groups: ...
26
votes
2answers
33k views

Calculating confidence intervals for a logistic regression

I'm using a binomial logistic regression to identify if exposure to has_x or has_y impacts the likelihood that a user will click ...
26
votes
1answer
10k views

Nonlinear vs. generalized linear model: How do you refer to logistic, Poisson, etc. regression?

I have a question about semantics that I would like fellow statisticians' opinions on. We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. The model ...
25
votes
3answers
40k views

Evaluating logistic regression and interpretation of Hosmer-Lemeshow Goodness of Fit

As we all know, there are 2 methods to evaluate the logistic regression model and they are testing very different things Predictive power: Get a statistic that measures how well you can predict the ...
25
votes
2answers
35k views

What's the difference between binomial regression and logistic regression?

I've always thought of logistic regression as simply a special case of binomial regression where the link function is the logistic function (instead of, say, a probit function). From reading the ...
25
votes
2answers
5k views

Why there are two different logistic loss formulation / notations?

I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$. Formulation/notation 1, $y \in \{0, +1\}$: $$ L(y,\...
24
votes
1answer
26k views

Help me understand adjusted odds ratio in logistic regression

I've been having a hard time trying to understand the use of logistic regression in a paper. The paper available here uses logistic regression to predict probability of complications during cataract ...
24
votes
1answer
6k views

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
24
votes
1answer
51k views

What is the difference between logistic and logit regression?

What is the difference between logistic and logit regression? I understand that they are similar (or even the same thing) but could someone explain the difference(s) between these two? Is one about ...
24
votes
1answer
51k views

Regression with only categorical variables

Is it possible to conduct a regression if all dependent and independent variables are categorical variables?
24
votes
3answers
21k views

How to know whether the data is linearly separable?

The data has many features (e.g. 100) and the number of instances is like 100,000. The data is sparse. I want to fit the data using logistic regression or svm. How do I know whether features are ...
24
votes
2answers
48k views

Adding weights to logistic regression for imbalanced data

I want to model a logistic regression with imbalanced data (9:1). I wanted to try the weights option in the glm function in R, but I'm not 100% sure what it does. ...
24
votes
2answers
2k views

Why do we model noise in linear regression but not logistic regression?

The canonical probabilistic interpretation of linear regression is that $y$ is equal to $\theta^Tx$, plus a Gaussian noise random variable $\epsilon$. However, in standard logistic regression, we don'...
24
votes
3answers
34k views

How to handle ordinal categorical variable as independent variable

I am using a logit model. My dependent variable is binary. However I have an independent variable which is categorical and contains the responses: ...
24
votes
2answers
25k views

What are the differences between Logistic Function and Sigmoid Function?

Fig 1. Logistic Function Fig 2. Sigmoid Function is it more like generalized kind of sigmoid function where you could have a higher maximum value?
24
votes
3answers
30k views

How to set up and estimate a multinomial logit model in R?

I ran a multinomial logit model in JMP and got back results which included the AIC as well chi-squared p-values for each parameter estimate. The model has one categorical outcome and 7 categorical ...
23
votes
11answers
20k views

Why is logistic regression called a machine learning algorithm?

If I understood correctly, in a machine learning algorithm, the model has to learn from its experience, i.e when the model gives the wrong prediction for the new cases, it must adapt to the new ...
23
votes
3answers
64k views

How to interpret main effects when the interaction effect is not significant?

I ran a Generalized Linear Mixed Model in R and included an interaction effect between two predictors. The interaction was not significant, but the main effects (the two predictors) both were. Now ...
23
votes
1answer
7k views

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares? It seems not clear to me because logistic loss and least squares loss are completely ...
23
votes
1answer
19k views

Understanding complete separation for logistic regression [duplicate]

Why does logistic regression not converge for a linearly separable data set? For linear separable data sets the model parameters go to infinity when mimizing the error function (according to ...
23
votes
1answer
14k views

Logistic Regression - Multicollinearity Concerns/Pitfalls

In Logistic Regression, is there a need to be as concerned about multicollinearity as you would be in straight up OLS regression? For example, with a logistic regression, where multicollinearity ...
23
votes
1answer
1k views

Should sampling for logistic regression reflect the real ratio of 1's and 0's?

Suppose I want to create logistic regression model which can estimate a probability of occurrence of some animal species living on trees based on characteristics of trees (f.e. height). As always, my ...
22
votes
3answers
31k views

How to compute the standard errors of a logistic regression's coefficients

I am using Python's scikit-learn to train and test a logistic regression. scikit-learn returns the regression's coefficients of the independent variables, but it does not provide the coefficients' ...
22
votes
1answer
17k views

interpreting estimates of cloglog logistic regression

Could someone advise me on how to interpret the estimates from a logistic regression using a cloglog link? I have fitted the following model in lme4: ...
22
votes
2answers
21k views

How to apply binomial GLMM (glmer) to percentages rather than yes-no counts?

I have a repeated-measures experiment where the dependent variable is a percentage, and I have multiple factors as independent variables. I'd like to use glmer from ...