1
$\begingroup$

My understanding is that logistic regression assumes a linear relationship between the logit of the outcome and each predictor variable.

I'm working on a case study from this MIT course. My model is making really poor predictions and I suspect it is because non-linearity.

idx <- sample(seq(1, 3), size = nrow(Book), replace = TRUE, prob = c(.45, .35, .2))
train <- Book[idx == 1,]
val <- Book[idx == 2,]
test <- Book[idx == 3,]

glm.fit1 <- glm(Florence ~., family = binomial, data = train)
summary(glm.fit1)
glm.probs1 <- predict(glm.fit1, test, type='response')
glm.pred1 <- rep("0",nrow(test))
glm.pred1[glm.probs1 >.5] <- "1"

This is the confusion matrix

> table(glm.pred1,test$Florence)

glm.pred1   0   1
        0 787  73
        1   0   1

How can I confirm that assumption?

What I've tried: I plotted the predictions against the log-transformed probabilities that came out of my model. I was told that doesn't work for poorly performing classifiers. Here is a post with info.

$\endgroup$
  • $\begingroup$ You said "logistic regression assumes a linear relationship between the logit of the outcome and..." -- well, no, not quite. It assumes a linear relationship between the logit of $p$ (which is the probability that the outcome is 1) and the predictors, not the logit of the outcome itself. $\endgroup$ – Glen_b -Reinstate Monica Jan 21 at 3:31
  • $\begingroup$ @Glen_b is correct but to be even more precise it assumes a linaer relationship between the logit of p and the continuous predictors. $\endgroup$ – StatsStudent Jan 21 at 3:33
  • $\begingroup$ I wondered whether to stress that I meant the predictors in the design matrix but figured that it wasn't necessary (perhaps wrongly, likely I should have done so). The linear predictor is clearly linear in $X$, since $\eta = X\beta$. $\endgroup$ – Glen_b -Reinstate Monica Jan 21 at 3:39
  • $\begingroup$ There are four primary methods that I use to assess the linearity of the relationship between the logit and continuous variables in logic regression. These are also the methods described, if I recall correctly in the Hosmer, Lemmeshow, and Sturdivant text on Applied Logisitc Regression (the diagnostics chapter). Check out the book for detailed examples and explanations. I may provide some additional details if time allows this evening or tomorrow. - Smooth scatterplots - Fractional polynomials - splines - Method of Design Variables $\endgroup$ – StatsStudent Jan 21 at 3:55
  • 1
    $\begingroup$ Can you tell us what your variables represent? And try a model with spline in the continuous variables $\endgroup$ – kjetil b halvorsen Jan 21 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.