1
$\begingroup$

While I was working on an exercise based this book, I discovered something interesting. When I fit a logit and simple linear probability model on the data (see code below), the predictions are almost identical. Intrigued by this, I decided to plot the data and discovered that the fitted lines almost overlap.

Is this pure coincidence or due to the nature of the data? I noted that the classes are not very well separated by the predictor, I assume this is the reason behind this? It would be great if someone could explain in which cases a logit model assimilates a simple linear probability model.

Here is the R code that I used:

library(ggplot2)
library(dplyr)
library(ISLR)

glm.fit <- glm(Direction ~ Lag2, data = Weekly, family = binomial)
Weekly$Direction <- as.numeric(Weekly$Direction)-1
lm.fit <- lm(Direction ~ Lag2, data = Weekly)

Weekly %>% ggplot(aes(x=Lag2, y=Direction)) +
  geom_point() +
  stat_smooth(method="glm", method.args=list(family="binomial"), color="#FF9999", se=FALSE) +
  geom_smooth(method=lm , color="steelblue", se=FALSE) +
  geom_hline(yintercept = 1, linetype="longdash") +
  geom_hline(yintercept = 0, linetype="longdash") +
  geom_text(aes(x = -15,y = 0.9, label = "Up")) +
  geom_text(aes(x = 10,y = 0.1, label = "Down")) +
  xlab("% Change two days before") +
  ylab("Direction") +
  theme_classic()

Linear Model blue, Logit Model pink

$\endgroup$
1
$\begingroup$

It's not coincidence. It's more to do with the nature of the logistic function and to some extent the nature of the data:

The second derivative of the logistic function at the place where $p$ is $\frac12$ is $0$. There's an interval around that where the logistic function is quite close to linear; in particular if the data are such that the proportion of $1$'s stays pretty much between $0.25$ and $0.75$, the difference between a linear fit and a logistic fit in that region will tend to be quite small.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.