Questions tagged [logistic]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
22
votes
1answer
32k views

Hessian of logistic function

I have difficulty to derive the Hessian of the objective function, $l(\theta)$, in logistic regression where $l(\theta)$ is: $$ l(\theta)=\sum_{i=1}^{m} \left[y_{i} \log(h_\theta(x_{i})) + (1- y_{i}) \...
22
votes
2answers
17k views

Building a linear model for a ratio vs. percentage?

Suppose I want to build a model to predict some kind of ratio or percentage. For example, let's say I want to predict the number of boys vs. girls who will attend a party, and features of the party I ...
22
votes
1answer
9k views

Omitted variable bias in logistic regression vs. omitted variable bias in ordinary least squares regression

I have a question about omitted variable bias in logistic and linear regression. Say I omit some variables from a linear regression model. Pretend that those omitted variables are uncorrelated with ...
22
votes
3answers
32k views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
22
votes
1answer
20k views

Logistic regression for time series

I would like to use a binary logistic regression model in the context of streaming data (multidimensional time series) in order to predict the value of the dependent variable of the data (i.e. row) ...
22
votes
1answer
2k views

Model selection with Firth logistic regression

In a small data set ($n\sim100$ ) that I am working with, several variables give me perfect prediction/separation. I thus use Firth logistic regression to deal with the issue. If I select the best ...
22
votes
1answer
2k views

Latent variable interpretation of generalized linear models (GLMs)

Short version: We know that logistic regression and probit regression can be interpreted as involving a continuous latent variable that gets discretized according to some fixed threshold prior to ...
21
votes
2answers
51k views

Significance of categorical predictor in logistic regression

I am having trouble interpreting the z values for categorical variables in logistic regression. In the example below I have a categorical variable with 3 classes and according to the z value, CLASS2 ...
21
votes
3answers
3k views

Is logistic regression a specific case of a neural network?

I ended up in a debate regarding logistic regression and neural networks (NNs). Is it wrong to say that logistic regression is a specific case of a neural network? I have seen a lot of explanation in ...
21
votes
3answers
37k views

Log-linear regression vs. logistic regression

Can anyone provide a clear list of differences between log-linear regression and logistic regression? I understand the former is a simple linear regression model but I am not clear on when each ...
21
votes
2answers
32k views

How to interpret coefficients from a logistic regression?

I have the following probability function: $$\text{Prob} = \frac{1}{1 + e^{-z}}$$ where $$z = B_0 + B_1X_1 + \dots + B_nX_n.$$ My model looks like $$\Pr(Y=1) = \frac{1}{1 + \exp\left(-[-3.92 + 0....
21
votes
3answers
14k views

From the Perceptron rule to Gradient Descent: How are Perceptrons with a sigmoid activation function different from Logistic Regression?

Essentially, my question is that in multilayer Perceptrons, perceptrons are used with a sigmoid activation function. So that in the update rule $\hat{y}$ is calculated as $$\hat{y} = \frac{1}{1+\exp(...
21
votes
1answer
13k views

Computing prediction intervals for logistic regression

I would like to understand how to generate prediction intervals for logistic regression estimates. I was advised to follow the procedures in Collett's Modelling Binary Data, 2nd Ed p.98-99. After ...
20
votes
3answers
14k views

Logistic regression or T test?

A group of persons answers one question. The answer can be "yes" or "no". The researcher wants to know whether age is associated with the type of answer. The association was assessed by doing a ...
20
votes
2answers
43k views

Plotting confidence intervals for the predicted probabilities from a logistic regression

Ok, I have a logistic regression and have used the predict() function to develop a probability curve based on my estimates. ...
20
votes
3answers
14k views

How does logistic regression use the binomial distribution?

I'm trying to understand how logistic regression uses the binomial distribution. Let's say I'm studying nest success in birds. The probability of a nest being successful is 0.6. Using the binomial ...
20
votes
2answers
5k views

What does the name “Logistic Regression” mean?

I am checking an implementation of Logistic Regression from here. After I reading that article, it seems the important part is the find the best coefficients to determine the sigmoid function. So I ...
20
votes
4answers
40k views

How should I check the assumption of linearity to the logit for the continuous independent variables in logistic regression analysis?

I am confused with the assumption of linearity to the logit for continuous predictor variables in logistic regression analysis. Do we need to check for the linear relationship while screening for ...
20
votes
2answers
10k views

Is there i.i.d. assumption on logistic regression?

Is there i.i.d. assumption on the response variable of logistic regression? For example, suppose we have $1000$ data points. It seems the response $Y_i$ is coming from a Bernoulli distribution with $...
20
votes
2answers
7k views

Least squares logistic regression [duplicate]

I have seen it claimed in Hosmer & Lemeshow (and elsewhere) that least squares parameter estimation in logistic regression is suboptimal (does not lead to a minimum variance unbiased estimator). ...
19
votes
8answers
2k views

Why is logistic regression particularly prone to overfitting?

Why does "the asymptotic nature of logistic regression" make it particularly prone to overfitting in high dimensions? (source): I understand the LogLoss (cross entropy) grows quickly as $...
19
votes
2answers
3k views

What is happening here, when I use squared loss in logistic regression setting?

I am trying to use squared loss to do binary classification on a toy data set. I am using mtcars data set, use mile per gallon and weight to predict transmission ...
19
votes
2answers
13k views

Classification with Gradient Boosting : How to keep the prediction in [0,1]

The question I am struggling to understand how the prediction is kept within the $[0,1]$ interval when doing binary classification with Gradient Boosting. Assume we are working on a binary ...
19
votes
1answer
19k views

Can glmnet logistic regression directly handle factor (categorical) variables without needing dummy variables? [closed]

I'm building a logistic regression in R using LASSO method with the functions cv.glmnet for selecting the lambda and ...
19
votes
2answers
4k views

How does the power of a logistic regression and a t-test compare?

Is the power of a logistic regression and a t-test equivalent? If so, they should be "data density equivalent" by which I mean that the same number of underlying observations yields the same power ...
19
votes
2answers
2k views

Does it make sense to use Logistic regression with binary outcome and predictor?

I have a binary outcome variable {0,1} and a predictor variable {0,1}. My thoughts are that it doesn't make sense to do logistic unless I include other variables and calculate the odds ratio. With ...
19
votes
5answers
4k views

Understanding which features were most important for logistic regression

I've built a logistic regression classifier that is very accurate on my data. Now I want to understand better why it is working so well. Specifically, I'd like to rank which features are making the ...
19
votes
1answer
11k views

How to fit a mixed model with response variable between 0 and 1?

I am trying to use lme4::glmer() to fit a binomial generalized mixed model (GLMM) with dependent variable that is not binary, but a continuous variable between zero ...
19
votes
3answers
44k views

Replacing Variables by WoE (Weight of Evidence) in Logistic Regression

This is a question regarding a practice or method followed by some of my colleagues. While making a logistic regression model, I have seen people replace categorical variables (or continuous variables ...
19
votes
2answers
18k views

Variable importance from GLMNET

I am looking at using the lasso as a method for selecting features and fitting a predictive model with a binary target. Below is some code I was playing with to try out the method with regularized ...
19
votes
2answers
14k views

Why use Platt's scaling?

In order to calibrate a confidence level to a probability in supervised learning (say to map the confidence from an SVM or a decision tree using oversampled data) one method is to use Platt's Scaling (...
19
votes
6answers
3k views

Linear regression or ordinal logistic regression to predict wine rating (from 0 and 10)

I have the wine data from here which consists of 11 numerical independent variables with a dependent rating associated with each entry with values between 0 and 10. This makes it a great dataset to ...
19
votes
3answers
2k views

Rare event logistic regression bias: how to simulate the underestimated p's with a minimal example?

CrossValidated has several questions on when and how to apply the rare event bias correction by King and Zeng (2001). I am looking for something different: a minimal simulation-based demonstration ...
19
votes
2answers
2k views

Updating classification probability in logistic regression through time

I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
19
votes
1answer
8k views

Plot and interpret ordinal logistic regression

I have a ordinal dependendent variable, easiness, that ranges from 1 (not easy) to 5 (very easy). Increases in the values of the independent factors are associated with an increased easiness rating. ...
18
votes
3answers
12k views

Which optimization algorithm is used in glm function in R?

One can perform a logit regression in R using such code: ...
18
votes
2answers
13k views

Calculate coefficients in a logistic regression with R

In a multiple linear regression it is possible to find out the coeffient with the following formula. $b = (X'X)^{-1}(X')Y$ ...
18
votes
2answers
13k views

Matrix notation for logistic regression

In linear regression (squared loss), using matrix we have a very concise notation for the objective $$\text{minimize}~~ \|Ax-b\|^2$$ Where $A$ is the data matrix, $x$ is the coefficients, and $b$ ...
18
votes
3answers
5k views

Model building and selection using Hosmer et al. 2013. Applied Logistic Regression in R

This is my first post on StackExchange, but I have been using it as a resource for quite a while, I will do my best to use the appropriate format and make the appropriate edits. Also, this is a multi-...
18
votes
4answers
28k views

Logistic regression with binary dependent and independent variables

Is it appropriate to do a logistic regression where both the dependent and independent variables are binary? for example the dependent variable is 0 and 1 and the predictors are contrast coded ...
18
votes
4answers
33k views

Interpretation of log transformed predictors in logistic regression

One of the predictors in my logistic model has been log transformed. How do you interpret the estimated coefficient of the log transformed predictor and how do you calculate the impact of that ...
18
votes
2answers
8k views

Why does logistic regression produce well-calibrated models?

I understand that one of the reason logistic regression is frequently used for predicting click-through-rates on the web is that it produces well-calibrated models. Is there a good mathematical ...
18
votes
2answers
12k views

Logit with ordinal independent variables

In a logit model, is there a smarter way to determine the effect of an independent ordinal variable than to use dummy variables for each level?
18
votes
1answer
2k views

Why are p-values often higher in a Cox proportional hazard model than in logistic regression?

I've been learning about the Cox proportional hazard model. I have a lot of experience fitting logistic regression models, and so to build intuition I've been comparing models fit using ...
18
votes
1answer
66k views

How to fix non-convergence in LogisticRegressionCV

I'm using scikit-learn to perform a logistic regression with crossvalidation on a set of data (about 14 parameters with >7000 normalised observations). I also have a target classifier which has a ...
18
votes
1answer
17k views

Logistic Regression : How to obtain a saturated model

I just read about the deviance measure for the logistic regression. However, the part that is called saturated model is not clear to me. I did an extensive Google search but none of the results ...
18
votes
1answer
3k views

Are robust methods really any better?

I have two groups of subjects, A, and B, each with a size of approximately 400, and about 300 predictors. My goal is to build a prediction model for a binary response variable. My customer wants to ...