Logit MLE in R and Finding the Odds-Ratio

Question

We know the Logit distribution is $\frac{exp(\beta'x_i)}{1+exp(\beta'x_i)}$

In R, if we want to execute a logit regression, we use:

glm.logit=glm(model,binomial(),Data.df)

This returns the relevant coefficients. How do we interpret these coefficients algebraically? I read another response which said that R returns the log odds, so each coefficient therefore has the interpretation of log$(\frac{p_i}{1-p_i})$? Where $p_i =\frac{exp(\beta'x_i)} {1+exp(\beta'x_i)}$ So p, the probability is equal to the distributional function? If this is correct, then all I need to do to get the odds ratio is take the exponential of, i.e. log$(\frac{p_i}{1-p_i})$ = $\beta'x_i$ $\implies$ $\frac{p_i}{1-p_i} = exp(\beta'x_i)$

in R:

exp(coefficients(glm.logit))

How would we calculate the logit model 'by hand'? Would you specify the logit distribution function, F(z) and then set z= linear model (OLS)?

Thank you

Answer 1

One way to find out what R has calculated is make a little experiment. Here I simulate data according to the model $$ \log\left(\frac{p_y}{1-p_y}\right) = b_0+ b_1x_1 + b_2x_2 $$

where I set b0=2, b1=-3, and b2=5. I let x1 and x2 be normally distributed.

N <- 10000
b0 <-  2
b1 <- -3
b2 <-  5
x1 <- rnorm(N)
x2 <- rnorm(N)
logodds <- b0+b1*x1+b2*x2
py <- exp(logodds)/(1+exp(logodds))
y <- rbinom(N,1,py)
coef(glm(y~x1+x2,family=binomial))
# (Intercept)          x1          x2 
#    2.029105   -3.072279    5.047870 

That result shows that the coefficients returned by glm are indeed the linear coefficients connecting the Xs to the log odds of Y.

As for calculating by hand, the glm function works differently from OLS. It maximizes a likelihood that depends on the distribution of error terms. In the case of logistic regression, the error terms are logistically (not binomially) distributed and that is quite different from Normal, as assumed by OLS.