2
$\begingroup$

I know about the disadvantages of linear probability models. However, while trying to understand glms I've stumbled over this:

When I estimate a linear probability model using the base lm() command, I receive different estimates than if I use the glm() command with family = binomial(link = "identity").

Here is an example:

 a <- rep(c(1,0), 1000)
 b <- rnorm(2000, mean = 4, sd = 2)

 lm(a ~ b)

Call:
lm(formula = a ~ b)

Coefficients:
(Intercept)            b  
   0.480940     0.004775

 glm(a ~ b, family = binomial(link = "identity"))

Call:  glm(formula = a ~ b, family = binomial(link = "identity"))

Coefficients:
(Intercept)            b  
   0.480954     0.004772  

Degrees of Freedom: 1999 Total (i.e. Null);  1998 Residual
Null Deviance:      2773 
Residual Deviance: 2772     AIC: 2776 

According to "An introduction to categorical data analysis" by Agresti, a linear probability model is a generalized linear model with binomial random component and identity link function. I do realise that R does not specify binomial(link = "identity") as a family object for models. However, it does not give an error or warning message when I use it.

So I have two questions:

  1. What does R do differently when it uses glm and not lm for estimating the linear probability model?

  2. Which of the two commands (glm or lm) should be used when estimating a linear probability model?

I know that the pragmatic answer to this question would be: "Why do you care about linear probability models?". However, I would like to understand the difference between the two commands for pedagogical reasons.

$\endgroup$
2
  • $\begingroup$ With "lm", without specifying "weights", you're actually performing OLS: rdocumentation.org/packages/stats/versions/3.6.2/topics/lm. The binomial random component is tipically used with a logit or probit (or cloglog) link, but it is possible to use a linear link, as you point out is said in the section "Linear probability model" of the book you quote. $\endgroup$ Commented Dec 3, 2022 at 23:04
  • 1
    $\begingroup$ As you say, you use the definition of linear probability model (LPM) given by Agresti. However, both by looking at the related Wikipedia page: en.wikipedia.org/wiki/Linear_probability_model and the references therein, and by googling it, it's easy to notice that, by LPM, we typically mean OLS, thus ignoring the binary nature of the outcome. Again quoting Agresti, the assumption of normality (however necessary only for exact inference: see my comment below) doesn't make sense, but in this way we prevent possible convergence issues by allowing for (nonsensical) value outside the 0-1 range. $\endgroup$ Commented Dec 4, 2022 at 14:00

1 Answer 1

2
$\begingroup$

You get different results because you have two different likelihood functions. The lm functions use, in glm language, a gaussian family, so the results are based on a normal-theory likelihood function. Your glm call with family = binomial(link = "identity") uses a binomial likelihood function, which implies a non-constant variance function. So some differences should be expected. The actual differences in your example is quite small, though.

$\endgroup$
4
  • 1
    $\begingroup$ Thank you, you are right! $\endgroup$
    – Nerd
    Commented Sep 30, 2020 at 11:16
  • 1
    $\begingroup$ Running glm(family=gaussian, link= "identity") gives the same results as the lm() command. I think this also answers my second question: I suppose it is better to use a binomial link function than a gaussian, since the dependent variable is binomial. $\endgroup$
    – Nerd
    Commented Sep 30, 2020 at 11:26
  • 1
    $\begingroup$ You need to be more careful with the terminology. There is no binomial linl or gaussian link, binomial and gaussian ... are familys, that is, they define the probability distributions used to construct the likelihood function. Link functions are logistic, identity, ... which determines the connection (transformation) between the linear predictor and the mean, that is, they determine the scale for the coefficients! It is interesting you found that the glm function accepts identity link for the binomial family, that works in your example with only categorical predictors ... else ? $\endgroup$ Commented Sep 30, 2020 at 13:20
  • $\begingroup$ I agree it's better to use a binomial random component (I follow kjetil b halvorsen in highlighting that a "binomial link" doesn't exist: both the cases you talked about use a linear link), to take the heteroskedasticity implied by the binary nature of the outcome into account: the OLS approach implied by "lm" without the "weights" option assumes homoskedasticity (normality is only necessary to get exact inference, not for having unbiased estimates: statisticsbyjim-com.translate.goog/regression/…) $\endgroup$ Commented Dec 3, 2022 at 23:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.