1
$\begingroup$

We know the Logit distribution is $\frac{exp(\beta'x_i)}{1+exp(\beta'x_i)}$

In R, if we want to execute a logit regression, we use:

glm.logit=glm(model,binomial(),Data.df)

This returns the relevant coefficients. How do we interpret these coefficients algebraically? I read another response which said that R returns the log odds, so each coefficient therefore has the interpretation of log$(\frac{p_i}{1-p_i})$? Where $p_i =\frac{exp(\beta'x_i)} {1+exp(\beta'x_i)}$ So p, the probability is equal to the distributional function? If this is correct, then all I need to do to get the odds ratio is take the exponential of, i.e. log$(\frac{p_i}{1-p_i})$ = $\beta'x_i$ $\implies$ $\frac{p_i}{1-p_i} = exp(\beta'x_i)$

in R:

exp(coefficients(glm.logit))

How would we calculate the logit model 'by hand'? Would you specify the logit distribution function, F(z) and then set z= linear model (OLS)?

Thank you

$\endgroup$
1
3
$\begingroup$

One way to find out what R has calculated is make a little experiment. Here I simulate data according to the model $$ \log\left(\frac{p_y}{1-p_y}\right) = b_0+ b_1x_1 + b_2x_2 $$

where I set b0=2, b1=-3, and b2=5. I let x1 and x2 be normally distributed.

N <- 10000
b0 <-  2
b1 <- -3
b2 <-  5
x1 <- rnorm(N)
x2 <- rnorm(N)
logodds <- b0+b1*x1+b2*x2
py <- exp(logodds)/(1+exp(logodds))
y <- rbinom(N,1,py)
coef(glm(y~x1+x2,family=binomial))
# (Intercept)          x1          x2 
#    2.029105   -3.072279    5.047870 

That result shows that the coefficients returned by glm are indeed the linear coefficients connecting the Xs to the log odds of Y.

As for calculating by hand, the glm function works differently from OLS. It maximizes a likelihood that depends on the distribution of error terms. In the case of logistic regression, the error terms are logistically (not binomially) distributed and that is quite different from Normal, as assumed by OLS.

$\endgroup$
4
  • $\begingroup$ I think your statements about the error terms in the logit and OLS case are mistaken. The error has a logistic distribution, and OLS does not assume normality. $\endgroup$ – Dimitriy V. Masterov Nov 3 '20 at 22:55
  • $\begingroup$ Fair enough on the logistic distribution, but for OLS, while calculation of the coefficients doesn't require normality, statistic inference does. But maybe I'm unnecessarily conflating the issues. $\endgroup$ – abstrusiosity Nov 3 '20 at 23:10
  • $\begingroup$ I think you need normality if you want to do MLE assuming normal errors or if you want to derive the finite sample distribution of the OLS estimator to do inference. If you are OK with asymptotic approximations, normality is not required for inference. $\endgroup$ – Dimitriy V. Masterov Nov 3 '20 at 23:17
  • $\begingroup$ The common assumption about normality in linear regression is that the error term is normally distributed in expectation. As an aside, while logistic regression can be motivated with an underlying latent variable which will have an error term, this is a fiction because error for a binary outcome can also only be binary. It's more natural to simulate realizations as binomial with probability equal to the inverse logit of the linear term (XB). $\endgroup$ – prince_of_pears Nov 3 '20 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.